By Dave LeeNorth America technology reporter
12 March 2018
"We can't assume that tech companies will find the solutions by themselves," Sadiq Khan told the BBC.
He said companies have to be "chivvied and cajoled to take action".
On Monday, he will share examples of abuse he has personally received.
The messages will form part of his keynote speech at the South by Southwest (SXSW) festival in Austin, Texas.
He said the messages could deter people from entering politics or public life.
"If someone like me is receiving these sorts of messages in a public environment," he said. "Imagine how you feel as a young person, if you're somebody who's putting your head above the parapet.
"You're going to think once, twice three times whether you want to do so."
'Heads in the sand'
The location of his speech is fitting – it was here where Twitter first gained popularity back in 2007. But in that decade, little has changed in the law.
"We have evolving economies, which means we should have evolving regulations.
"For too long politicians and policy makers have allowed this revolution to take place around us and we've had our heads in the sand."
In his speech he will reference rules put in place in Germany that allows fines of up to 50 million euros if hate speech is not removed in a timely fashion.
"Germany is an example of where the German government said 'Enough. Unless you take down hate messages, unless you take down fake news, we will fine you'.
He added: "I want to work with the tech companies, but you have to be responsible."
Trump tweets
Mr Khan acknowledged that the rate of his abuse has been accelerated by tweets from President Donald Trump that mentioned the Mayor specifically in the wake of a London terror attack.
In November, Mr Trump retweeted posts from Britain First, a far-right group that has now been banned from the platform.
"If you are the most powerful man in the western world, and you are amplifying messages from far-right groups, Britain First, that has an impact on many people. That group incites hatred.
"President Trump has lots of followers and some of them have shown interest in me.
"I'm a reluctant participant in any 'verbal fisticuffs' between the President of the USA and me.
"But I've got a responsibility as the Mayor of the most diverse city in the world to speak up for my residents."
Stricter regulation
But in talking at this extremely pro-tech event on this topic, the Mayor treads a fine line.
Raising the possibility of stricter regulation, even fines, risks London gaining a reputation as being an anti-start-up city – a place where doing business on the internet could be riskier financially than in other parts of the the world.
"I want London to be the centre of disruptive technology, and businesses thinking of starting up, scaling up or giant tech companies coming to London," Mr Khan said.
"But, and there's a very important but, we as politicians and policy makers have got to work with you to make sure it works for everyone.
"What I don't want is a situation where we end up where Germany's ended up where because their citizens don't feel protected they're taking this sort of action. Germany is an example of what happens if companies don't self-regulate."
In a related attempt to clamp down on behaviour by some tech firms, Mr Khan said recent suggestions to tax tech companies on revenue, rather than profits, was a "really interesting debate that is finally happening".
But he said he believed it was cooperation between countries that would have the most impact on how tech firms approached their tax affairs.
"What we want is joined-up-ness," he said.
"What we don't want is companies using an army of accountants to find the loopholes."