Apostle St Matthew the Evangelistʼs church in Anykščiai is a fine example of the 19−20th century Lithuanian sacred architecture in the neo-Gothic style. Designed by a Russian architect, Nikolay Andreyev, who worked as an engineer in Kaunas governorate and was the architect to most neo-Gothic churches built in Lithuania, the church of Anykščiai leaves its visitors deeply impressed with its architectural forms of solemn and convivial nature, let alone its size. The church is the highest two-towered church building in Lithuania, reaching 79 metres in height. The church is constructed in a rectangular plan, with a nave and two aisles (all of the three in the form of a hall) as well as a presbytery and a pentagonal apse. There are three entrances to the building in the façade. Over the front door to the church there is a wimperg (a triangular shield). On both sides of the wimperg visitors can see a blind-arcading, a series of small, narrow, whitewashed, pargeted arches. In the centre of the facade there are three windows displaying the form of an equilateral arch. The facade is further divided into narrow gothic niches.

The church towers assume a shape of a square at the base, while moving upwards, they grow into an octagon. In the centre of the wall of each tower, an equilateral-arch window is nestled and elegant pilasters are constructed at the four corners of the two towers. The pilasters are then surmounted by small pinnacles. The square-shaped part of each tower is terminated with a pediment at the top, where it is further broken into Gothic niches. The same pediments found at this part of the two towers are simulated at the top of the octagonal spires, yet here they grow slightly more pointed than those found beneath. The current steeples look different from those available in 1909 as they suffered a severe damage in WWI, when they were completely destroyed in explosion. In 1917 temporary wooden structures were built, and it took more than a decade for the destroyed steeples to be finally replaced with the brick ones in 1928−1929. The reconstructed steeples are now actually 3 metres lower and simpler than those originally designed. Along with the church wall built in the 20th century, the gate and the storehouse, the church forms a solid ensemble which has been declared an immovable cultural value of the local region.

While inside the church, visitors can see a few monuments which once belonged to the old Baroque church. Made by the local masters, there are two Baroque confessionals placed in the aisles. Both of them are decorated with Christian symbols and designs of folk art performed in the technique of incrustation. The sacristy houses two canvases of the Baroque period too, namely "The Crucifixion" and "St Laurence".

Over the years, an impressive collection of fine arts belonging to the 20th century has been compiled in the church. Thus funded by writer Antanas Vienuolis, in 1957 a bust to Antanas Baranauskas, the author of the well-known Lithuanian poem "The Forest of Anykščiai", was designed and carved in granite by sculptor Henrikas Rudzinskas. Whether placed inside the church or in the churchyard, lots of art works came to contribute the treasury of the church in the late 20th century, when one of the most outstanding Lithuanian clergymen, Monsignor Albertas Talačka, worked there. With effort made by the Monsignor in 1971, over the portal of the church there was constructed a stained glass of thick glass lumps called "St Matthew the Evangelist". Specially designed by artist Marija Anortė Mackėlaitė for this particular church, it was the only stained glass of this kind found in Lithuania at the time. Later in 1971-1988, 15 other stained glasses of a thin layer were added by the same artist to the church windows in the clerestory, now letting some playfulness of colours and light into the church on a sunny day.

With another bust in bronze, designed by sculptor Jonas Meškelevičius, the memory of the famous linguist, educationalist and Jesuit monk, Konstantinas Sirvydas, born in the local region, was honoured when in 1982 it was unveiled in the church. The same year a commemorative plaque to another outstanding figure from the region, priest and poet Klemensas Kairys was put on the wall inside the church. It is a bronze low relief designed by sculptor Leonas Žuklys. Decorated with an elaborate design, this historic marker was carved from wood by local folk artist Liudas Tarabilda.

While furnishing the sanctuary, Monsignor Talačka aimed to avoid art works which would merely copy the dull, rigid forms, once popular in Christian art, yet now seen as lacking in vitality. He highly valued the art based on authentic religious and aesthetic experience which would be well familiar with Christian iconography and symbolism and at the same time could be described as modern and related to the contemporary mind and aesthetics of the present. In the years of the Soviet regime, when sacred art was held in disgrace, the Monsignor was surprisingly bold to seek young artists who would be able and willing to create authentic art of this kind. This was how a few paintings combining tradition with modernism by Vaidotas Žukas came to the church. Two of them called "St Casimir" and "Blessed Archbishop Jurgis Matulaitis" now hang on the walls of the two aisles, while the third painting in an original composition, radiant with the sacred to convey the essence of the baptism sacrament, is found in the baptistry.

Yet the most valuable masterpiece that the church boasts about is one of the most impressive works belonging to Lithuanian sacred art of the second half of the 20th century – Stations of the Cross, else known as Way of Sorrows, made by artist Rimas Idzelis for the chapels in the wall surrounding the churchyard in 1982–1988. It was Monsignor Albertas Talačka, who cared and arranged for the Stations to be erected in Anykščiai church. The fifteen polychrome reliefs, that the Stations consist of, are all carved from wood to tell the story of the Passion of Jesus. Meant to describe and commemorate the final hours of Jesus, the Stations reveal the drama of the event as well as a wide diversity of emotions ranging from sorrow, passion, fear and evil to sympathy, sacrifice and love. By intrinsic expression and ability to elevate the described event to a mysterious dogma conveying symbol, the Stations continue the tradition of sacred art, once known to flourish in the periods of Gothic and early Renaissance.

Some other works of art that can be a subject of interest to the visitors are also found in the churchyard. Among them, a few sculptures – original pieces of sacred art - created by professional sculptors are placed on the graves of the buried priests to honour their memory. Here, for instance, artist Romas Kazlauskas made an excellent use of the potential of expression that white marble could offer in carving a statue called "Our Lady of Sorrows", which now stands on the grave to late Fr Jonas Uogintas. White as it is and distinguished by a compact silhouette as well as laconic plastic, the figure lends itself to a suggestion of unspeakable chastity and intrinsic suffering, all found in stone. On the grave to late Fr Alfonsas Gražys a bronze statue "Ecce Homo" (or else known as "Man of Sorrow") by Vladas Vildžiūnas then brings a visitor to the personality of Jesus. As a monumental figure of expressive design the statue comes to combine royalty with humility, the divine with human, peace and light with nobility, all present in the suffering.

Worth of the visitor's attention is a unique stone cross created as a headstone to Canon Jurgis Žitkevičius by the self-taught local sculptor, Algis Ladiga. On the pedestal to the cross the author carved two armorial cartouches in 1978. One of them shows six hosts while the second is an image of St George, which well reminds of Vytis (the Chaser), the Lithuanian coat of arms. Both of the cartouches are meant to be symbols of gratitude to Lord and homeland. The decision to use the Vytis resembling St George's icon on the pedestal was fairly drastic at the time.

The churchyard is also host to other commemorative monuments, yet these are less distinctive. At the gate to the churchyard, for example, there are two ornamented wooden crosses. Made by local folk artists, they were built in 1987 and 1988. The former replaced the old cross to missionaries whereas the latter, created by folk artist Alvydas Seibutis, was meant to mark sufferings of local deportees to Siberia. Another composition carved in stone which has been hosted at the back of  the churchyard since 1986 is "The Holy Family", a monument to the parents of locally born celebrated figures. Designed by sculptor Antanas Kmieliauskas, the memorial is the only commemorative monument of this kind in Lithuania.