They are based on the popular urban legend of alligators living in New York City. Not much is known about Giant Alligators in GTA IV. The only mention of the myth is in a Liberty Tree article.
Most sightings of Giant Alligators in Liberty City can be found in the sea and the city's Subway systems, but there are also large sewage grates that are believed to be hiding something inside. The Giant Alligators are very aggressive and will attack the player on sight, and due to their size, will usually be able to kill the player fairly quickly. There is no clear evidence or pictures to support this myth, but it being mentioned in the game makes a slight possibility that they could exist in the water all around Liberty City and Alderney. Nevertheless, there is a small group of myth hunters who do believe that alligators exist in the game, and they have reported that these creatures will attack unprovoked. Unfortunately, due to lack of evidence, this myth is considered false.
1920 Sewer Alligators
There's another mention of alligators on an article on Craplist, in which a man says that he is looking for a dwarf crocodile named Betty, which he brought from Vice City. This is a reference to the 1920 incident about Sewer Alligators in New York City. Both Vice City and Liberty City are based on Miami and New York. Therefore, making it an easter egg for GTA Vice City and GTA IV.