The Museum Folkwang, founded in Hagen by Karl Ernst Osthaus in 1902, was the first museum of contemporary art in Europe. The most significant works were transferred from Hagen to Essen in 1922, from which point on, aside from a period when the National Socialists temporarily divested the collection, the museum was able to pursue a high level of collecting activity. Today, it is one of the most high profile museums of Classic Modernism in Germany.

The new museum extension by David Chipperfield Architects Berlin complements the original listed building, preserving its integrity while perpetuating the architectural principle with an ensemble of six volumes and four inner courtyards, gardens and covered walkways. The publicly accessible areas connect seamlessly with the existing exhibition areas. A generous open stairway leads from Bismarckstraße into the new foyer, which takes the form of an open interior courtyard with a restaurant and a bookstore, and is protected from the street by a glass façade.

Visitors are welcomed into a succession of differing rooms – exhibition areas with ceiling heights of up to six metres, a library and reading room, a multifunctional room, an events space, storage and restoration workshops. The extension, which is oriented towards Essen’s city centre, provides a new urban focus together with the neighbouring Institute for Advanced Study in the Humanities. The translucent, alabaster-like façade consists of large rectangular recycled glass slabs. The colour of the façade shifts with the changing natural light and the integrated window openings sit flush with the façade. Polished screed was used for the floors, which is similar in colour and texture to the concrete elements used for the plinth.

The Folkwang is a museum that doesn’t have to shout. It has a choice collection of Van Goghs, Gauguins, Cézannes and Rodins, together with ancient curiosities. It has works by Warhol, Rothko, Pollock and other bankable post-war artists, plus strong collections of posters and photography. The medium-sized industrial city of Essen is not famous as a cultural Mecca, but work like this ensures that long queues gather on the approach ramp.

Folkwang’s new building was made possible by a single grant from the Alfried Krupp von Bohlen und Halbach Foundation at an early stage in its development, meaning that it didn’t have to make much of a noise.

It makes no attempt to ingratiate, exaggerate or advertise. It is stripped, spare, simple. Its ancestry is Mies van der Rohe and before him Karl Friedrich Schinkel, but without the fanatical glint you get in Mies. It is more sane and reasonable, like the well-balanced, well-made modernism that West Germany sponsored in the 1950s and ’60s, by architects like Egon Eiermann.

Museum Folkwang is based on the collection of Karl Ernst Osthaus who, a century ago, was a leading collector of what was then contemporary art in the nearby town of Hagen. After his death the museum continued his pioneering spirit until the coming of the Nazis, who denounced half the work as degenerate, and sold it off. After the war a few of the dispersed objects were recovered and the collecting of new art resumed.

This history has created a collection that is eclectic and personal, with Egyptian, Islamic, Japanese and Indonesian artefacts. Yet its Western works, especially those of the post-impressionists, are up there with the very best. It is a collection that might allow you to pick out a linear narrative, from Manet to Warhol and beyond, but which is better experienced in a more diffuse or wandering way.

David Chipperfield Architects has responded with a series of pavilions and courtyards, allowing different routes through the collection. All exhibition space is on one level such that, as they say, you don’t have to think about the means by which you are getting about the building. Offices are above, other service spaces below.

The courts, inaccessible to the public, present oblongs of immaculate grass to be seen but not touched. Glass walls open up views to parks and villas in the semi-suburban surroundings, creating a sense of interpenetration between the museum and the neighbourhood. There is also an easy connection with the 1950s gallery, to which the larger new building is technically an extension. The older building, like the new, is spare, with glazed courtyards; but for some variations in detail and updated technology, their spirit is almost identical.

The architects say they wanted it to be like certain 19th-century museums, with ‘huge rooms, where anything can happen.’ The prevailing atmosphere is calm.

The architects also say they wanted to get ‘straight to the art, without any preamble,’ so the entry route leads quite directly, with shop and café on either side, to a reception in the centre of a big gallery-like space. Art is on show here, albeit currently a single Andreas Gursky, and you do indeed feel you are already in the heart of the museum.

All of which prepares the ground for the art itself, some of which is spectacular. It is almost, but not quite, as if the architecture is ideally absent: as if as little barrier as possible is desired between art and city. Not quite, because the building does have its own kind of robustness and presence. The ambition seems rather to be as good as possible at simple-but-difficult things that define an experience: light, height, proportion, placement, movement.

The building does indeed do these things extremely well, the only crashingly awful moments coming with illuminated signs announcing the museum’s Vincent & Paul restaurant. These succeed in making the museum seem an adjunct of the restaurant, and shatter the poise of the rest of the building. Whoever is responsible for such stupidity should go and pursue a career in shopping malls, rather than museums.

But this raising-up is a matter of a few metres. The pavilions, with their linking glass passages, seem at first sight to belong to a well-bred post-war school or business park.

It is only in their faint classicism, and their cladding, that the pavilions give a hint of their cultural content. The cladding is recycled glass, greenish, slightly rough on the surface, and placed in horizontal panels such that it is faintly reminiscent of marble or stone. The panels are covered at the corners by L-sectioned glass cover strips, which enhance the impression of solidity. It is a murmur of differentiation.

Such plainness brings to mind Charles Jencks’ mockery of Mies van der Rohe’s Illinois Institute of Technology, in his 1977 book The Language of Post-Modern Architecture. Why, asked Jencks, should the chapel look like a boilerhouse and a boilerhouse look like a chapel? And why did critics, he asked, overlook these obvious questions and focus instead on esoteric interpretations of corner details? To apply such arguments to the Folkwang, should it not announce itself more explicitly as a museum? And should its role not include a more positive contribution to the surrounding streetscape?

This is similar to a feeling that other recent Chipperfield projects inspire, such as Barcelona City of Justice law complex (AR July 2009): couldn’t it have been a bit more charming? Certainly it’s possible to imagine a building that would give more externally while still being a wonderful place to see art. But it’s also the case that what Chipperfield calls ‘rigour’ has its uses. The consistent plainness of the architecture creates both a calm atmosphere, and a sense of a place apart, in which the art can shine.