For decades, the task of reforming Britain’s complex welfare state has proven a gargantuan and elusive undertaking. With politicians still grappling with the difficulties of rolling out ‘universal credit’ and setting the eligibility requirements for free school meals, it is no surprise that the system proves yet more baffling to members of the public.

Nowadays, we tend to view Britain’s welfare state as the product of post-war central planning. Yet, as exemplified by the difficulty in creating functioning welfare programmes even today, such complicated institutions take time to emerge, as Christopher Renwick’s timely history of the Welfare State (recently released in paperback) reminds us.

His analysis begins in the mid-19th century with Victorian reformers and continues until the creation of the National Health Service in 1948. Along the way, Renwick crafts a fascinating narrative, part intellectual and part political history, detailing who thought what, when and why.

The cast of names that emerges in the book’s opening chapters is unsurprising – social surveyors like Seebohm Rowntree and Charles Booth make appearances, as does their work diagnosing and attempting to cure what would later be termed the five giants of “Disease, Want, Ignorance, Idleness and Squalor”. At this point in history, Renwick suggests, the real sticking-point was not under-funding of benefit schemes but the widespread belief that living in poverty was caused by moral failure, rather than any societal inequality.

One of the book’s most illuminating portions is on the subject of ‘degeneration’, a concept that emerged in response to Darwin’s theory of natural selection – and the impact on degeneration theory on welfare provision. Fears of social, racial and cultural decline abounded in the late 19th century, with adherents positing that human beings could slide down the cosmic scale of advancement “making the descendant far simpler or lower than its ancestor”.

Yet degeneration theory became increasingly influential, argues Renwick, following the British Empire’s poor performance in the Boer War against Dutch Settlers. Arnold White, an anti-Semitic and eugenicist journalist who had already written a book called The Problems of a Great City, which advanced arguments that would certainly get him barred from public office if expressed today. White was convinced that the difficulty of victory in Africa was due to the physical and mental deterioration of the British soldier. “It was no surprise Britain had struggled against the Boers,” he noted, “because the raw material the country had to work with was clearly not up to the job.”

His blunt solution was for those “who failed to meet the army’s requirement” to “be stopped from having children… by forced sterilisation if necessary”. Such ideas garnered support across the political spectrum, from socialists like Karl Pearson to the popular novelist and Fabian HG Wells.

For Renwick, the fact that mass sterilisations of the working poor never occurred seems attributable more to electoral cynicism than strong moral objections from the governing classes. Arthur Balfour’s administration in 1903, he writes, had “no interest in turning the working classes against the Tories”, and instead looked to welfare programmes to address the humiliations of the Boer War. Readers might wonder how different society might have looked if the Representation of the People Act 1884, granting suffrage to some 60% of working age men, had been introduced a mere two decades later.

The last two-thirds of the book leave the trappings of East London poverty and Toynbee Hall and immerse themselves in Westminster, examining the progress of the nascent welfare state through the fortunes of political parties and politicians. Starting from Winston Churchill’s observation, in the wake of the First World War, that the power of the state to coordinate the large-scale manufacture of munitions in a timely and effective fashion represented the “greatest argument for State Socialism ever produced”, we follow power transitions between the Liberals, Conservatives and Labour Party.

The post-war consensus, he notes, was informed by burgeoning Keynesian economic theories suggesting that mass unemployment in the wake of World War II could be addressed by proper economic policy.

Perhaps most interestingly, Renwick details how, long before the outbreak of World War II, Britain had one of the most extensive welfare states in the world, built not, as is commonly believed, by pressure from socialist activists alone – but by sometimes unlikely allies, drawn from the establishment. Neville Chamberlain, now entrenched in popular legend as the PM who failed to stand up to Hitler, emerges as a passionate social reformer. Renwick illustrates his campaigns in the 1920s to decrease the state pension age by five years and his attempts to reform the Poor Laws and replace it with a geographically consistent service, funded through general taxation. He also sets out how the Beveridge report of 1942 captured the beleaguered public’s imagination – making the Conservatives’ decision to vote against the creation of the NHS on the bill’s third reading an electorally disastrous move.

But the path from Victorian philanthropists like Seebohm Rowntree to politicians like Nye Bevan – which at its core is what this book is about – can best be seen in the preamble to the Bevan’s infamous invective, where he ascribes the poverty of his youth not to a Victorian weakness of spirit, but policy failure. Speaking at a 1948 rally in Manchester, Bevan explained that “no amount of cajolery, and no attempts at ethical or social seduction, can eradicate from my heart a deep burning hatred for the Tory Party that inflicted those bitter experiences on me.”

Going on to utter words that would go on to adorn thousands of Labour supporters’ T-shirts and banners, he told the crowd: “So far as I am concerned, they are lower than vermin.”

Bread For All: The Origins of the Welfare State by Christopher Renwick is published by Penguin Books. £9.99, 323pp.