While Madison Square Garden had prepared itself for the presence of the German Bund, many around New York City considered the Nazi sect less welcome in their city. About 100,000 anti-Nazi protesters gathered around the arena in protest of the Bund, carrying signs stating "Smash Anti-Semitism" and "Drive the Nazis Out of New York".[6] A total of three attempts were made to break the arm-linking lines of police, the first of these, a group of World War One Veterans, wrapped in Stars and Stripes, were held off by police on mounted horseback, the next, a "burly man carrying an American flag" and finally, a Trotskyist group known as the Socialist Workers Party, who like those before, had their efforts halted by police.[4]

If you gather a crowd of 100,000 counter-protesters, several times larger than your own rally, not sure how 'popular' you are.

This is not what OP claimed.

Well the US was very much in love with the nazi party until it became politically inexpedient. Then they pretended they never were but didn't actually change anything

While being popular and then having that popularity decline was part of it, they suggested that the reason it became unpopular was because that support became politically impractical. They also suggest that the US itself, not US citizens, were in live with the Nazi party. This may be an accident due to poor phrasing, but assuming that's what they were going for, their sources only show of a small political activist group, not any governing body.

Also, the group, although the size isn't actually reported anywhere among the sources I could find, was actually pretty small, and was mostly German immigrants who were torn between supporting their homeland and their new home. This was made more difficult a decision due to German propaganda calling for people of German descent to stand together.

Precise membership figures are not known. Estimates range from as high as 25,000 to as low as 6,000. Historians agree that about 90 percent of Bund members were immigrants who arrived in America after 1919. In Wisconsin, the most heavily German state, the Bund seems to have mustered barely 500 members, which would rule out the possibility of anywhere near 25,000 members nationwide.

Assuming that the largest reported member count of 25,000 members was correct, that's hardly popular. The US had a population of 139 million people in 1945. This would be 0.0018% of the population. To put that number into perspective, ~12 million Americans were in military service, about 9% of the American population at the time. So the people willing to risk their lives to kill nazis outweighed this political activist group by 5000%