Peter Paul Rubens, with Jan Brueghel the Elder, “Madonna on Floral Wreath,” c. 1619
“And a crowd was sitting about him; and they said to him, ‘Your mother and your brothers are outside, asking for you.’ And he replied, ‘Who are my mother and my brothers?’” (Mark 3:32-33)
Mark 3:31-35 (RSV) And his mother and his brothers came; and standing outside they sent to him and called him.  And a crowd was sitting about him; and they said to him, “Your mother and your brothers are outside, asking for you.”  And he replied, “Who are my mother and my brothers?”  And looking around on those who sat about him, he said, “Here are my mother and my brothers!  Whoever does the will of God is my brother, and sister, and mother.”
Matthew 12:46-49 While he was still speaking to the people, behold, his mother and his brothers stood outside, asking to speak to him.  But he replied to the man who told him, “Who is my mother, and who are my brothers?”  And stretching out his hand toward his disciples, he said, “Here are my mother and my brothers!”
Luke 8:20-21 And he was told, “Your mother and your brothers are standing outside, desiring to see you.”  But he said to them, “My mother and my brothers are those who hear the word of God and do it.”
James Spencer Northcote (1821-1907): an Anglican priest who was received into the Catholic Church, commented on these passages in his series of meditations, written in the 1850s: The Life of Mary in the Gospels (The Mother and Brethren of Jesus):
We are quite at liberty to imagine, if we like, that Our Lord, after uttering the words which the Evangelists have recorded, rose up and proceeded to grant His Mother the interview she had asked for; there would be nothing at all strange in such a supposition; on the contrary, it is more possible than not; but it is not certain. All that we are told is that He answered the interruption in these words, “Who is My mother and My brethren? And then looking round about on them who sat about Him, He saith, Behold My mother and My brethren. For whosoever shall do the will of God, he is My brother, and My sister, and mother.”
I need not say that these words were not really an answer sent to His mother and brethren, but rather a lesson of instruction addressed to those “who sat about Him;” nor can it be necessary to point out to anyone who is familiar with the Gospels, how common a thing it was with our Blessed Lord to direct His answers not so much to the questions that had been put forward, as to the inward thoughts and motives of those who put them; how sometimes He set aside the question altogether as though he had not heard it, yet proceeded to make it the occasion of imparting some general lesson which it suggested. This is precisely what He does now.
Jesus took this opportunity to show that He regarded all of His followers (in what would become the Christian Church) as family. Similarly, He told His disciples, “I have called you friends” (Jn 15:15). It doesn’t follow that this is “a rebuff of this kin” (i.e., his immediate family).
He simply moved from literal talk of families to a larger conception and vision of families as those who do “the will of God.” Thus, Jesus habitually used “brethren” to describe those who were not His immediate family:
Matthew 5:47 And if you salute only your brethren, what more are you doing than others? Do not even the Gentiles do the same?
Matthew 23:8 But you are not to be called rabbi, for you have one teacher, and you are all brethren.
Matthew 25:40 And the King will answer them, `Truly, I say to you, as you did it to one of the least of these my brethren, you did it to me.’
Matthew 28:10 Then Jesus said to them, “Do not be afraid; go and tell my brethren to go to Galilee, and there they will see me.”
Luke 22:32 “but I have prayed for you that your faith may not fail; and when you have turned again, strengthen your brethren.”
John 20:17 Jesus said to her, “Do not hold me, for I have not yet ascended to the Father; but go to my brethren and say to them, I am ascending to my Father and your Father, to my God and your God.”
Bible searches reveal that “brethren” is used 191 times in the New Testament, mostly in this sense. So is “brother” (116 times in the New Testament). “Sister” is also used in the epistles, referring to fellow Christians who are female:
Romans 16:1 I commend to you our sister Phoebe, a deaconess of the church at Cen’chre-ae,
1 Corinthians 7:15 But if the unbelieving partner desires to separate, let it be so; in such a case the brother or sister is not bound.
Philemon 1:2 and Ap’phia our sister and Archip’pus our fellow soldier, and the church in your house:
James 2:15 If a brother or sister is ill-clad and in lack of daily food,
Arguably, all of this might be thought to have started in the words of Jesus here under consideration. It’s simply the beginning of the Body of Christ, and the Christian Church being regarded as one large, extended family.
Lastly, Jesus refers to His own mother as the mother of St. John, when He asked His disciple to watch over her after Jesus’ death:
John 19:26-27 When Jesus saw his mother, and the disciple whom he loved standing near, he said to his mother, “Woman, behold, your son!”  Then he said to the disciple, “Behold, your mother!” And from that hour the disciple took her to his own home.
And of course, we have the long tradition of calling priests (in Catholicism and Orthodoxy) “Father.” Female leaders of nuns and religious are called “Mother”; for example, Mother Teresa; now St. Teresa of Calcutta, or Mother Angelica, who founded EWTN. Monks are called “Brother” and nuns, “Sister,” etc.