Legislation to create the new holiday on August 11 was enacted after passing through the upper house with the support of a majority of ruling and opposition lawmakers, a parliamentary official said.
It came after the Japanese Alpine Club and other mountain-related groups lobbied for the bill, claiming that Japan -- where Shintoism's animistic beliefs have shaped the culture -- needed to celebrate its hills.
Japan already marks Marine Day, which is sometimes translated as Ocean Day, on the third Monday of July.
The legislation states that the day is designed to share "opportunities to get familiar with mountains and appreciate blessings from mountains".
A large chunk of Japan's landmass is mountainous, and walking or trekking is a popular pastime, particularly among older Japanese.
The mountains also bless Japan with excellent skiing throughout the winter, with foreign visitors raving about the quality of the snow.
Once Mountain Day takes effect from 2016, Japan will have 16 official holidays a year, the highest tally among the Group of Eight major powers, and double the number observed in Britain.
And, unlike Britain, where many of the days off are known only as "bank holidays", each of Japan's rest days celebrates something specific.
These include Children's Day, Coming of Age Day, Constitution Day and National Foundation Day.
Japan has steadily added to its total of days off in recent decades, along with switching from a six-day working week to five days as the country's economy has matured.
This is at least in part an effort to tame the tendency particularly prevalent among office workers to put in long hours and not to take time off, with observers saying the reluctance is borne of an unwillingness to burden colleagues with extra work.
While Europeans tend to take long summer holidays, in which they will use up two or more weeks of their annual leave at once, many Japanese limit their time away to extended weekends.
It is not unusual for employees to forgo up to half of their annual leave every year, although this is changing among younger Japanese.
Japanese employees worked an average of 2,031 hours a year in 1990, compared with 1,831 in the United States and 1,578 hours in Germany, according to the Japan Institute for Labour Policy and Training.
The number, however, declined nearly 15 percent to 1,728 hours in 2011, falling below those in the United States, Italy or New Zealand, according to the government-backed organisation.