Working hard at a job, traveling for business and trying to move up the career ladder can make doing everything else in life that much tougher.

Unless, of course, you have some flexibility in how you structure your job -- especially in terms of when and where you work.

And that's exactly what workers say they want, according to a new EY survey of 9,700 full-time employees in eight of the world's largest economies: the United States, the United Kingdom, Germany, Brazil, Mexico, India, China and Japan.

They listed flexibility as a top feature they want in a job -- ranked just behind competitive pay and benefits. And they cited a lack of it as among the top reasons they would quit.

Related: The 4-day workweek is real ... for employees at these companies

The same survey found that a third of workers globally say it's gotten harder to manage their work-life balance in recent years. Employees in Germany (49%), Japan (44%) and the United Kingdom (37%) were most likely to report this.

At the same time, workers in Germany (70%) were the most likely to report that they had flexible work schedules, followed by 61% of workers in the United States and India.

Countries where workers are least likely to have flexible schedules are China (22%) and Japan (30%).

Sometimes, simply having work flexibility as an option isn't necessarily a guarantee of employee happiness. A full 67% of all survey respondents listed "flexibility stigma" at a company as a big reason why they'd quit.

In other words, opting to telecommute a day or two a week or work flexible hours is great, unless the end result is that you're denied a promotion or raise because your boss doesn't support the company's policy.

Related: No. 1 cause of bad work-life balance? Bad bosses

Among U.S. respondents, 9% said they had actually "suffered a negative consequence as a result of having a flexible work schedule." Among U.S. millennials, nearly 15% said they had. Those negative consequences included losing a job, being denied a promotion or raise, being assigned less interesting or high-profile assignments, or being reprimanded.

More than other generations, U.S. millennials were most likely to say they valued flexibility. And if they get it they said they are more likely to recommend the company as a place to work, be more engaged and work longer hours.

The EY survey found that half of all managers in the United States, Mexico, India and Brazil say they're working more than 40 hours a week. Those in India, Brazil and Mexico were the most likely to have seen an increase in their hours in the past five years as were close to a third of U.S. managers.

A majority of all U.S. respondents, meanwhile, said they'd change jobs, change careers or even be willing to give up a promotion to achieve better work-life balance.