In that regard, a point often overlooked is that the Hebrew word NEPHIYL--translated "giants" in the KJV, actually extends far beyond the context of great physical size. Besides physical giants such as the sons of Anak, and Goliath and his brothers, the word NEPHIYL can also refer simply to men of great notoriety--"mental giants," or "larger-than-life personalities--usually in a negative context, such as a bully or tyrant. We're told that the children born to the sons of God and daughters of men became the "mighty ones" in the earth, "men of renown." The same Hebrew word for mighty ones, (GIBBOWR), is used to describe Noah's great grandson, Nimrod--the first great secular monarch named in the Bible:
And Cush begat Nimrod: he began to be a mighty one in the earth.
He was a mighty hunter before the LORD: wherefore it is said, Even as Nimrod the mighty hunter before the LORD. (Gen 10:8-9)
From the text, we can conclude that there were strong, ungodly men before the Flood, just as Nimrod typified such individuals after the Flood. "There were giants in those days, and also after..." (Gen 6:4)
So remarkable was the hunting prowess and charisma of this celebrated ruler, that his name became proverbial with great hunting skill--just as Einstein's name today is synonymous with scientific genius. ("That guy is a regular Einstein"). Yet, we should note carefully that the Hebrew word translated "before" in Gen 10:9, can mean against, or in opposition to. In that context, Nimrod can be seen as a rebellious figure, who opposed God and his purposes, leading the people of his kingdom away from God--both a catalyst and symbol of rebellious, ungodly mankind.
Seeing that the common language of man was uniting man toward evil intent, God somehow divided human speech into separate languages, effectively dispersing mankind throughout the earth into separate nations, according to that group's tongue. He destroyed the unification of mankind because it was united against Him--rallying around the joint project of constructing the tower of Babel. This tower was meant to "reach unto heaven," intended to be a refuge from any future flood God might send. (This clearly demonstrates an ignorance of the ungodly of God's promise to never again send such a flood). The tower itself, by it's very intent, became a lasting symbol of man's rebellion against God.
The Hebrew word nephiyl conveys the idea of being "cast down" or "fallen." This is used by some to suggest that the sons of God were fallen angels--which is reasonable and logical. But let's not forget the fallen spiritual condition of man--the result of choosing the same rebellion against God's authority that Satan and the fallen angels had: unsaved mankind and the rebellious angels share the same destiny because they share the same depraved, fallen nature. If we view the "sons of God" as the faithful people of God, in contrast to the ungodly majority upon the earth, it's not hard to see that term as denoting those descendants of Adam who remained faithful to God in the midst of ungodliness and the worship of false gods.
In Genesis 4:25-26, we learn that, in the time of Adam's grandson Enos, people began to call themselves by the name of the Lord, apparently referring to the first organized congregation of the Lord. We can assume that, from that time until the Flood, the Lord had a small group that called themselves the Lord's people, and who lived holy, distinguishing themselves from the ungodliness around them--as do true saints today.
Fast forward to the time immediately prior to the Flood: the descendants of Adam had multiplied greatly in number, nearly all choosing the path of Cain into unrighteousness and rebellion against God. We read that the whole world of that time was steeped in violence and wickedness, and grew worse with each generation.
Now, let's suppose that, in the years preceding the Flood, even godly men began to fall away from God's precepts, and began to take multiple wives from among the beautiful but ungodly heathen women--perhaps painting their eyes and faces, and dressing in a provocative way, like ancient Jezebels. And suppose these men were lured farther away from God by the licentious pagan religions of these women and began to worship their false gods and godesses. Consider the "sons of God" marrying the "daughters of men" in that context: even those who once maintained the faith of Abel, Seth and Enoch, had abandoned God as a result of embracing the idolatrous practices of pagan wives. This would explain the bearing that intermarriage had on the increase of sin and corruption as implied in the text--without necessitating the involvement of angels, or any other supernatural being. This trend of ever-increasing corruption would certainly explain why God saw the world as hopelessly decadent, and a lost cause:
God saw how corrupt the earth had become, for all the people on earth had corrupted their ways. So God said to Noah, "I am going to put an end to all people, for the earth is filled with violence because of them. I am surely going to destroy both them and the earth. (Gen 5:17-18 NIV)
This same spiritual infidelity--which God equated to whoredom, would later be manifested repeatedly among the Hebrews. It was because of the danger of such corruption that God had forbidden His people to intermarry with the heathen nations, clearly warning them of the inevitable consequences (Deut 7:1-4). Their refusal to heed God's warnings eventually culminated in their going into Gentile captivity--Israel, into Assyria, and then Judah, into Babylon. Since they insisted on following after the gods of foreigners, God allowed them to fall under the oppressive yoke of foreigners. Indeed, it was the influence of pagan women that caused the spiritual downfall of Solomon. He foolishly sought to appease them by building structures for them to honor their false gods and goddesses--using the very wealth God had blessed him with!
So then, a case can easily be made that the sons of God were not angels, but instead, godly men who had departed from the faith--accentuating the corruptible nature of man, and hastening the decision of God to "amputate" the ungodly members of humanity in order to preserve a godly seed from which he could prepare and train a people to receive His Redeemer. This people would eventually come out of Abraham, a direct descendant of Noah through Shem, born some 10 generations later--and possessing the same unwavering faith in God that Noah had.
As far as physical giants suggesting angelic fatherhood, we might note that Robert Wadlow (1918-1940), was just one inch shy of being 9 feet tall--probably not much shorter than Goliath. Wadlow's uncommon size has been attributed to a pituitary gland disorder, and To my knowledge, neither he nor his father has ever been suspected of being anything other ordinary human beings. Additionally, it's not uncommon to see equally mortal men today, who stand 7 feet tall, or taller.
So then, I think we can explain everything about Genesis 6 without the sons of God being supernatural beings. We cannot rule out the influence of malevolent spirits in the affairs of mankind (no pun intended), but with regard to the intermarriage and corruption described in Genesis 6, it seems unnecessary for angels to have fathered any children. Moreover, while open to debate, it seems unlikely that God would create eternally-existing spirits with unnecessary biological functions for physical procreation. At any rate, this issue has always been, and will no doubt continue to be, a matter of speculation and debate. And that's fine. Discussion and comparing views is how we learn.