“The New Colossus” has a very concrete and specific setting: Liberty Island, which lies in the Upper New York Bay. Liberty Island is a very small island that’s located where the mouths of the Hudson River and the East River come together. The speaker references these two rivers in lines 3–4:

Here at our sea-washed, sunset gates shall stand
A mighty woman with a torch

The “sea-washed, sunset gates” refers to the Hudson and the East River, and their confluence marks the place where there “shall stand / A mighty woman with a torch”—that is, the Statue of Liberty. The speaker offers another clue to the poem’s setting in line 8, mentioning “the air-bridged harbor that twin cities frame.” The “twin cities” referenced here are likely New York City (i.e., Manhattan) and Brooklyn, which were still distinct cities in Lazarus’s lifetime. Though separated by the East River, here the speaker describes the two cities as a “frame” that centers the “air-bridged harbor” between them. It is precisely there that the New Colossus stands on her dedicated island. The speaker’s elevated and idealized description of New York Harbor ultimately enshrines Liberty Island as a doorway meant to welcome immigrants to the New World.