In 1950 Hiroshima City, gambler Shoji Yamanaka is involved in a fight over cheating in a card game and stabs several men. He is sentenced to two years in prison, where he befriends Shozo Hirono, who is already serving time for murder. Following Yamanaka's release from prison, he visits a restaurant but can't pay his bill. The owner, Yasuko, offers to let him eat for free, but when he insists on paying her with his watch, a gang led by Katsutoshi Otomo, the rebellious son of yakuza patriarch Choji Otomo, beat him severely before the fight is stopped by the senior Otomo. Swearing to take revenge on his attackers, Yamanaka accepts an offer from Boss Muraoka, Yasuko's uncle, to join his yakuza clan. Yamanaka is assigned to the family of Muraoka's sworn brother, Kunimatsu Takanashi, until he is chased out of Hiroshima by Muraoka for beginning a romantic relationship with the widowed Yasuko; Takanashi arranges for him to temporarily join a family in Kyushu.

Yamanaka redeems himself a year later by carrying out a hit for the yakuza family sheltering him, and Muraoka welcomes him back as a formal member in a ceremony witnessed by his associate, Kanichi Tokimori. Katsutoshi is kicked out of the Otomo Family by his father for trying to muscle in on Muraoka's territory and forms his own family with Tokimori's support in a bid to take over Hiroshima. In retaliation, Muraoka expels Tokimori, prohibiting him from conducting any business in the city. Tokimori flees to Kure, while Katsutoshi and his men spark a war by attacking the Muraoka family offices.

In Kure, Hirono's former boss Yoshio Yamamori pays him and his small family to protect Tokimori while he works to resolve the dispute. However, when Yamanaka shows up in Kure with orders to kill Tokimori, Hirono makes his own decision to assassinate Tokimori as a means of both settling the dispute and protecting his friend's honor. The Otomo Family gains new leadership, and Muraoka reconciles with them on the condition that Katsutoshi dissolve his family. Muraoka also gives his blessing for Yamanaka to pursue Yasuko, only to then call on him to kill three of Katsutoshi's men who were planning to assassinate the Otomo Family chairman and reignite the war. Yamanaka is arrested by waiting policemen for the murders and is sentenced to life in prison.

Takanashi, also held for a minor offense, informs Yamanaka that Muraoka is coercing Yasuko into marrying her deceased husband's brother. Yamanaka manages to escape from prison by feigning illness. When he attempts to assassinate his boss, Muraoka suggests that Takanashi may have lied. Yamanaka pleads for another opportunity and offers to eliminate Katsutoshi. Although he only injures him, Katsutoshi is later arrested and sentenced to life imprisonment. His deputy is fatally stabbed after meeting with Muraoka, and the remaining members are jailed. The Otomo Family, weakened by internal strife, is absorbed into the Muraoka Family. Muraoka orders Yamanaka to kill Takanashi for his betrayal but also alerts the authorities, who are ordered to shoot Yamanaka on sight and seal off the entire neighborhood to trap him. Cornered, Yamanaka takes his own life with his gun. At his funeral, Hirono glares angrily at Muraoka, while Yamamori and other mourners praise Yamanaka for preserving his boss's honor.

In fall of 1963, the police crack down on yakuza activities nationwide as the government prepares for the 1964 Summer Olympics. However, the war between the Yamamori Family and the Shinwa Group against an alliance of the Uchimoto, Hirono, and Akashi Families rages on. Bosses Noburo Uchimoto and Shozo Hirono, along with Akashi lieutenant Shinichi Iwai, recruit two additional bosses, Hidemitsu Kawada of the Kawada Family and Tomoji Okajima of the Gisei Group, to their side. One of Hirono's officers is killed by men from the Makihara Family, run by Yamamori's underboss, and Hirono vows to take revenge himself, but others, including his advisor Kenichi Okubo, urge him not to. Akira Takeda, captain of the Yamamori Family, threatens Okubo's life if he doesn't keep Hirono in Kure, as Yamamori has fled to Hiroshima City while he and Makihara continue the war.

The police, knowing that the gangs are spending heavily to bring in reinforcements from across Japan, begin putting the squeeze on their rackets and business interests. When an Uchimoto Family member accidentally kills a drunken civilian, the public demands further action and newspapers begin running stories attacking and demonizing the yakuza. The police put a constant stakeout on Hirono's office, effectively paralyzing him since he is still technically out on parole. The cowardly Uchimoto refuses to take action, while the Akashi Family becomes distracted by their own problems in Tokyo. When Hirono learns that Yamamori will be traveling to Kobe, he secretly leaves his base in disguise planning to kill him personally. However, during the trip, the two men escorting him run off intending to perform the hit themselves. The Akashi Family intercepts them, not wanting to be dishonored by letting Yamamori get killed on their turf. Iwai concocts a plan: he and Hirono arrange a large memorial service for his murdered officer, using it as an excuse to bring in hundreds of men for an attack on Hiroshima City.

Uchimoto is abducted at gunpoint by Takeda and Yamamori and rats on his partners. Yamamori tips the police off to a year-old crime Hirono committed to have him arrested. His arrest forces the attack to be called off, and Yamamori safely returns to Kure so he can crush Hirono's weakened family. After learning that Takeda is having Okajima's girlfriend spy on him, he tricks her into revealing Okajima's location so he can be assassinated, angering Takeda. Shoichi Fujita, a senior Gisei Group member and Okajima's likely successor, retaliates by bombing the headquarters of Yamamori officer Shoichi Eda, and Uchimoto sells out his own men when they decide on a whim to kill Yamamori as a favor to Takeda. Their plan foiled, Uchimoto's men start a public brawl that turns into a shootout, giving the police a convenient excuse to arrest Uchimoto, Yamamori, and several other bosses. Iwai and his men immediately fly to Kure to rebuild the Gisei Group and expand the Akashi Family's influence, while Takeda forms a coalition of Hiroshima yakuza to fight them. He orders a bombing of Boss Akashi's house in Kobe to frame the Shinwa Group, triggering retaliation from the Akashi Family and leading to more violence and arrests.

Kawada, tired of the fighting and wanting to protect his own territory from the Gisei Group, manipulates Yazaki, a low-ranking member of his family, into killing Fujita; Yazaki is caught and sentenced to twenty years for murder. Iwai visits Hirono in jail and explains to him all that has happened, before adding that the Akashi Family is pulling its support as the police have forced them to make peace with the Shinwa Group. Prosecutors build a case against Hirono and his parole is revoked, earning him seven years in prison. Makihara gets three years, Eda five, Yamamori a year and a half (and his wife informs him that the government is willing to buy out one of his businesses for a billion yen after he's released), and Uchimoto cuts a deal to dissolve his family and retire from the yakuza, which gets him off on probation. While waiting in a cold, unheated room to be booked into prison at the film's end, Takeda half-jokingly tells Hirono that he might turn his family into a political committee when he gets out.