The Nephilim are the children of Demons and Angels. Ancient Nephilim were the first humans on Earth, and were typically at odds with the alius found there. Once, the Nephilim were considered more powerful than either Angels or Demons. However, after the bloodline was diluted and thinned, their prowess diminished in time.
The term Nephilim is now used to describe the children of at least one Angelic parent. In Cole's instance, his father is a Fallen Angel. Despite his father being considered a Demon now, he still has the blood of an Angel and thus his son is referred to as a Nephilim.
Due to their power, both Demons and Angels feared the Nephilim. However, they are now an extremely rare occurrence, with the Angels having abandoned the mortal realm and their war with the Demons.