Harari calls the agricultural revolution "History's Greatest Fraud." Wheat domesticated humans, and humans, on average, wound up a lot worse off. Hunter gatherers are exposed to the elements and to predation by large carnivores, but they have a lot of compensations. They eat a highly varied diet, and a nutritious one, because it's the diet millions of years of evolution prepared us for. Because they move constantly, they reproduce slowly, which retards population growth. Since they live in small isolated groups, without domestic animals, they are relatively free of disease. For most, a few hours of work each day suffices to provide them with food and other requirements.
Most people in agricultural societies, by contrast, eat a very narrow diet, live in crowded villages, side by side with their own filth, and reproduce rapidly. Agriculture typically requires long hours of literally backbreaking labor - slipped and herniated disc's become common only after the invention of agriculture. The crowded conditions and accumulated waste make disease rampant, and when animals are added to the mix they bring our epidemic diseases.
So why did humans let themselves be enslaved by wheat? Harari's answer, given in far more detail, basically boils down to the same way we make bad decisions today: it seemed like a good idea at the time. A little labor invested in planting some wheat grains rather than just waiting for nature to provide paid dividends at the end of the season. As population exploded, it took more and more labor to produce enough grain to feed all the children. And now returning to an HG lifestyle was impossible, population growth, as Harari put it, had burned Humanity's boats.
Of course agriculture made possible civilization and all it has brought, but for most of 12,000 years it was a very bad deal for the average agriculturist.
He's not fond of the deal animals get from being domesticated either, and devotes a few pages to the cruel methods used then and now.
He offers some alternative scenarios for human domestication by wheat, but I found them less persuasive.
Sapiens: A Brief History of Humankind by Yuval Noah Harari