The ends result of this survey must remain meagre and disappointing to both defenders and opponents of the doctrine of the virginal conception. Taken by itself, historical-critical research simply does not have the sources and tools available to reach a final decision on the historicity of the virginal conception as narrated by Matthew and Luke. One's acceptance or rejection of the doctrine will be largely influenced by one's own philosophical and theological presuppositions, as well as the weight one gives to Church teaching.Lüdemann [Jesus After 2000 years, 122-24] concludes that we can extract as a historical fact behind Matt 1.18-25 the existence of a hostile rumour about the illegitimacy of Jesus. Lüdemann suggests that rape by an unnamed man, possibly even a Roman soldier, is the most likely explanation. According to Crossan’s analysis what we have here are strata 1 traditions that are multiply attested. If we are to be consistent with that, merely asserting these traditions have their genesis in dogmatic imaginations doesn’t persuade. Raymond Brown notes where these traditions agree:
They agree on these points: Chap. 1 deals with the prebirth situation; chap. 2 with the birth or postbirth situation. The parents of Jesus are Mary and Joseph, who are legally engaged or married but have not yet come to live together or have sexual relations. Joseph is of Davidic descent. There is an angelic announcement of the forthcoming birth of the child. The conception of the child by Mary is not through intercourse with her husband but through the Holy Spirit. There is a directive from the angel that the child is to be named Jesus. The roles of Saviour (Matt 1:21; Luke 2:11) and Son of God (Matt 2:15; Luke 1:35) are given to Jesus. The birth of the child takes place at Bethlehem after the parents have come to live together. The birth is chronologically related to the reign of Herod the Great (Matt 2:1; Luke 1:5). Eventually, the child is reared at Nazareth.Brown elaborates further on this question by noting that:
We are left with more questions than answers. But clearly, there are historical elements which have been reworked through a scripture framework [“History Scripturized” cf. Goodacre] so as to relay the significance or theology of these historical peculiarities. But clearly, the overlapping of agreements, despite the divergences, constrain our conclusions in such a way as to exclude a genesis in pure authorial imagination. As Nolland concludes, Despite all critical reserve the traditional view continues to have much to commend it [See Nolland, Luke, 1:42-48].
Such a general judgment need not imply that there are not some historical elements in either or both accounts. The mutual agreement have an importance, for they probably represent points that were in a tradition antedating both Matthew and Luke. For instance, an intelligent case can be made that Jesus was truly descended from David and born at Bethlehem in the reign of Herod the Great. Arguments to the contrary are far from probative (Brown 1977: 505–16). In particular, the virginal conception (popularly but confusingly called the Virgin Birth) should be evaluated cautiously. Despite extremely limited attestation and inherent difficulties, no satisfactory nonhistorical explanation which could dispense with the virginal conception has been brought forward. The frequent approach to the virginal conception as a theologoumenon, whereby the common “Son of God” title of Jesus would have been translated into a (fictional) narrative in which he had no human father, could acquire plausibility only if there were a good antecedent or parallel for the idea of virginal conception. There is no good antecedent or parallel. While there were Greco-Roman and other examples of male gods impregnating earth women to produce a divine child, the NT contains no hint of such a sexual union. Within Judaism there was no expectation that the messiah would be born of a virgin. (The MT of Isa 7:14 does not clearly refer to a virgin, and even the LXX need mean no more than that one who is now a virgin will conceive through future intercourse. Matthew has not derived Jesus’ conception from Isa 7:14, but interpreted the OT passage through Christian data.) A claimed Hellenistic-Jewish tradition that the patriarchal wives conceived from God without male intervention (Philonic allegory; Gal 4:23, 29) is far from certain. (On all this, see Boslooper 1962; Brown 1977: 517–33). In terms of historical catalysts behind the concept of a virginal conception, those worth noting are: (a) the agreement of Luke (implicit) and Matthew that Jesus was conceived before Joseph and Mary came to live together and hence that the birth might be noticeably early after cohabitation; (2) the 2nd-century Jewish charge that Jesus was illegitimate (Or. Cels 1.28, 32, 69), possibly reflected earlier in John 8:41.
[Brown, Infancy Narratives, ABD.]