Mr Xi said "achieving total unity" was the "collective hope of all Chinese people" and any attempts to divide it were "doomed to fail".
The speech was a strong warning against any attempt at separatism from places like Taiwan and Hong Kong.
Mr Xi also warned China could not be complacent about its development.
Meanwhile, Premier Li Keqiang used his once-a-year news conference to say China was committed to global co-operation on trade.
He said China would further open up its economy and "ensure that both domestic and foreign firms" were "able to compete on fair terms in China's large market". Chinese officials have made similar pledges in the past.
The premier's comments are a marked contrast to the recent protectionist rhetoric and threats of tariffs from the Trump administration in the US.
'Punished by history'
Xi Jinping became president in 2013 and now looks likely to lead China indefinitely, after the National People's Congress (NPC) – a rubber-stamp parliamentary session that meets once a year – voted to remove a two-term limit on the presidency from the constitution.
He is now considered the most powerful Chinese leader since Mao Zedong, and has further cemented his position by moving many of his key allies in to leadership roles during the NPC.
In his speech in the Great Hall of the People, Mr Xi highlighted major achievements of Chinese history like the invention of gunpowder and the writings of Confucius.
He laid out his own grand vision for China, saying history had proven that "only socialism can save China".
Chinese people, he said, "have the spirit of fighting the bloody battle against our enemies to the bitter end".
Pointing at China's global ambitions, he said his country would be strong but not aggressive and would not develop itself at the expense of the rest of the world.
But he warned that China had to be united to prosper, and it was "absolutely impossible to separate any inch of our great country's territory from China".
"All acts and tricks to separate the country are doomed to fail and will be condemned by the people and punished by history."
The comments were a clear warning to places like Taiwan, which is self-ruling but considered by Beijing to be a a rebel region that must be reunited with the mainland – by force if necessary.
The issue remains a major potential flashpoint. Last week US President Donald Trump angered Beijing by singing a new law which encourages the sending of top officials to Taiwan to meet their counterparts there, and vice-versa.
There have also been vocal calls for greater autonomy or even independence for Hong Kong in recent years – including massive street protests – and China says it is fighting an Islamist separatist insurgency in Xinjiang province.
Dissenting voices or protests are not tolerated at the NPC, which brings together 3,000 delegates who are theoretically elected but in reality handpicked by the government.
Among other major changes this year, the NPC confirmed a major shake-up of ministries, introducing several new bodies overseeing areas like the environment, army veterans and culture.
Key positions including the central bank head and chief economic advisor were appointed, and the NPC ratified a law to set up a new powerful anti-corruption agency, far extending the reach of Mr Xi's flagship campaign.
The anti-corruption drive has seen more than a million officials disciplined, but is seen by some observers as a tool to sideline his political rivals.
Though it was overwhelmingly backed by parliament, the decision to lift the presidential term limits has been criticised both abroad and within China.
In the run-up to the NPC, former state newspaper editor Li Datong wrote to lawmakers that scrapping term limits for the president and vice-president would sow the seeds of chaos.
"This is against the tide of civilisation and won't stand the test of time," he told BBC Chinese. "It will be considered a farce in Chinese history in the future."