What Leeds United discovered last night was the fundamental difference between an upset and a humiliation, a difference that even the historical and welcome unpredictability of knockout football cannot hide.

* CLICK HERE TO WATCH A PICTURE SLIDESHOW OF MATCH ACTION FROM THE GAME.

United's players would have liked to have taken their leave of Don Valley Stadium blaming their exit from the Johnstone's Paint Trophy on the inspiration afforded to last night's opponents, Rotherham United, but the aftermath of a 4-2 defeat was a time for inward examination rather than outward praise.

The evening was Rotherham's to savour after their highest-scoring victory over Leeds for half a century, and Gary McAllister sensibly left acknowledgment of their win to his counterpart Mark Robins. The only issue of relevance to United's manager was the careless loss his club had sustained.

Leeds attacked with enough fluency and imagination last night to match Rotherham's four goals, denied such a tally only by the crossbar and Rotherham's evergreen goalkeeper Andy Warrington, but their defensive ineptitude was deservedly exposed by a conclusive defeat.

The extent of the failure of the defensive line selected by McAllister was almost indescribable, and it was the deciding factor in a game where Rotherham plundered four of the most routine goals ever.

Three of those were converted directly from set-pieces, and the fourth was scored by Mark Hudson after United's captain, Andrew Hughes, was drawn into conceding a penalty two minutes before half-time. It was, for Rotherham, an open invitation to the quarter-finals of the trophy's northern section.

There are certain defeats which have hidden value or strands of positivity running through them, and there are others which do nothing more than make a manager cringe. Last night's loss fell into the second category.

McAllister made sweeping changes to the team which played at Peterborough on Saturday, calling on eight of his fringe players, but what was an opportunity to impress did nothing more than push a number of individuals who have been overlooked for recent league matches further away from the top of the pecking order.

There are unlikely to be many survivors when Leeds play Brighton at Elland Road this weekend, a game which many in McAllister's camp appear to be defining as must win and certainly a fixture they cannot lose. In their own surroundings, a repeat of the performance seen in Sheffield last night would not be tolerated or remotely desirable.

Clear skies prevailed in South Yorkshire yesterday afternoon, and the atmosphere around Don Valley – a stadium with towering floodlights and an open ambiance not dissimilar to that of Dynamo Dresden which Leeds so nearly visited last year – was more in tune with a pre-season friendly.

The same could be said of United's disorganised defending and also the peculiar charade which ensued around the goalposts at either end of the field in the hour before the game was scheduled to start.

United's players walked onto the pitch in their tracksuits after arriving at the stadium and were immediately drawn into a discussion beneath one crossbar, indicating with their raised arms that the goal fell below regulation height. Rotherham's ground staff took to the field with a tape measure and began anxiously re-sizing the posts, apparently to the satisfaction of Leeds.

It was not clear whether the size of the goals would have been a deal-breaker in terms of the match kicking off, but the problem reflected the makeshift nature of Rotherham's surrogate home, a venue designed originally for athletics and which succeeded Millmoor as the club's ground before the start of this season.

It stretches the imagination to describe Don Valley as a football stadium – three quarters of the arena stood empty last night, partly due to the low profile of the Johnstone's Paint Trophy but also because Rotherham lack the necessary safety certificate to open more seating areas – but their team have enjoyed a surge in credibility which continued yesterday.

A victory over Grimsby Town last Saturday wiped away the remainder of the 17-point penalty imposed on Rotherham at the behest of the Football League in August, and their exploits in the Carling Cup had accounted for Sheffield Wednesday, Wolverhampton Wanderers and Southampton. It seemed from the moment that Rotherham scored after 18 minutes that Leeds would be added to that list.

Robins' attack had made little impression on United's defence before that point but a direct approach into the visitors' box had more effect on the unit in front of David Lucas.

Ian Sharps, Rotherham's central defender, read the direction of a long-throw in from Andy Nicholas and met the delivery with a deft flick of his head, guiding the ball over Lucas who had strayed from his line and was unable to adjust his position.

The goal was ridiculously soft and the result of a yard of space afforded to Sharps, but while McAllister's backline castigated themselves, his team as a whole were more inclined to feel aggrieved.

Robert Snodgrass had hit the crossbar in the seventh minute with a left-footed free-kick, the pace of which left Warrington struggling to cover his goal, and Rotherham's keeper met a curling shot from Andy Robinson three minutes later with a one-handed save. Lucas had not been examined until Sharps' opportunistic header found the net at his far post.

An appeal for a penalty from Rotherham's players followed immediately when Drewe Broughton fell to the floor as Rui Marques fought to direct a defensive header towards Lucas, and though referee Scott Mathieson gave Marques the benefit of considerable doubt, he was persuaded to point to the spot two minutes before half-time.

Leeds by then had recovered from their early concession with a goal after half-an-hour, created by a superbly disguised pass from Neil Kilkenny which Prutton thudded against Warrington's palms and Jonathan Howson finished off from six yards out, but they proved porous again on 43 minutes.

Reuben Reid took hold of possession 20 yards from Lucas' box and cut through a stream of Leeds players, forcing Hughes to slide in with a desperate attempt to win the ball which clipped Reid's trailing ankle.

Lucas read Hudson's feeble penalty, parrying the ball low to his left, but Hudson composed himself and met the rebound with a sharp side-footed shot which slipped past United's prostrate keeper. Again, the misfortune seemed to be McAllister's. Kilkenny's raking shot had whistled beyond Warrington's goal by the slimmest of margins two minutes earlier, and Sharps' defensive nous put him in the right position to block a shot from Snodgrass inside the box after a quick attack from the visitors.

The signs of the first half indicated that it would not be United's night, a feeling that three Championship clubs have experienced at Don Valley before Leeds this season. Broughton's goal three minutes into the second half only deepened that suspicion.

Leeds' marking was again open to criticism as Broughton left Marques standing in United's area and headed a cross from Dale Tonge to the left of Lucas, and the goal appeared to have sparked an implosion.

Seven minutes later, Marques was out-jumped again by Nick Fenton, who crashed Alex Rhodes' corner into the net as an astonished McAllister looked on.

Rotherham's mammoth lead was lessened slightly in the 56th minute when Enoch Showunmi ran from a position which looked to be offside and drove home a shot at the near post with the help of a deflection off Warrington's body, but Rotherham's goal refused to give in for a third time.

Luciano Becchio appeared from the bench and hit the bar with a header, and the excellent Warrington pulled off crucial parries from Snodgrass, Kilkenny and Howson.

In amongst the chaos, Pablo Mills had a header disallowed and Alan Sheehan – another of United's three substitutes – walked from the field with a hamstring strain, leaving Leeds a man down and ready to depart a competition they have never won and would be happy to avoid for years to come after this season.