English-speaking Christians often think of the God of the Bible as the one and only 'god' there is. Within the English-speaking world, this is more or less true, because the English-speaking world uses a particular definition of the word 'god'.
To English-speakers, the word 'god' almost always carries the meaning of 'omniscient, omnipotent, omnipresent', and since most English-speakers are raised in a world where Christian (or Jewish, or Muslim, or deist, or Christian-ish) monotheism is the standard concept of 'god', it is generally thought that 'god' can only refer to the God of the Bible, Yahweh (though most simply know him as 'God' and not 'Yahweh', or 'Jehovah', or 'YHWH'), and hence, the God of the Bible is omniscient, omnipotent, and omnipresent.
In ancient Hebrew (and hence, within the world of the ancient writers of Scripture), this is not what 'god' meant.
Their word for 'god' (elohim) most literally means something like 'mighty one', and is regularly applied to entities who are not Yahweh. And, regularly, these applications are not intended to be ironic or facetious. In Scripture: pagan deities are called 'elohim'; Yahweh's angels are called 'elohim'; humans rulers are called 'elohim'; spirits of dead humans are called 'elohim'. Only idols seem to be called 'gods' in an ironic tone, but this is primarily because of their nature as being inanimate objects (e.g. clay, wood, stone, or metal). Their inanimateness is what qualifies them as being 'false' gods, not because they are not Yahweh.
Carrying on into the New Testament, the Greek word for 'god' (theos) picks up where the Hebrew word left off. It is, again, occasionally used ironically of idols. The one time where the New Testament (apparently) specifically derides these other gods as false gods is when Paul speaks of 'those who are called gods'. But he then follows this up with 'there are many gods and many lords'. His exact meaning is unclear, and could really go either way.
The Scriptural concept of 'gods' (elohim or theos) referred to a wide range of things, but the thing that they have in common is not that they were worshiped. Yahweh, pagan deities, angels, and spirits of dead men belonged, in some way, to a 'world' beyond this one. Human rulers are sometimes called 'elohim' because they were either seen to be physically descended from gods (a very pagan concept) or had been divinely appointed by the gods (Israel's judges were said to be 'gods' because they had been appointed under Yahweh's authority).
Within the Scriptural usage of the word 'god' (elohim or theos), there are many gods, and their existence was beyond question. But whether these other gods exist as real deities (I personally find this doubtful), or are simply 'fallen' angels (maybe), or humans with great power and authority, is not determined within Scripture.
But what is to ultimately be taken away is that within the Scriptural usage of the word 'god' (elohim or theos), while 'there are many gods and many lords', there is only one true, absolutely supreme God, which is Yahweh, revealed through Jesus. He is not the one and only true God because he is simply called 'god', but because he demonstrates his absolute superiority to all other gods (real or not) through his actions, as particularly shown and emphasized in Scripture.