The defender was sent off for two needless bookings in the first half and incurred the wrath of his manager after the game
The England international, who started the game at centre-back, picked up his first caution for needlessly charging down Joe Hart's goal-kick before hacking down James Milner before half-time.
That meant United had to play the remaining 52 minutes with 10 men and a makeshift back four of Antonio Valencia, Michael Carrick, Paddy McNair and Luke Shaw after Marcus Rojo also exited the game after dislocating his shoulder.
Sergio Aguero ultimately made the extra man count by firing home a 63rd minute winner for City, though United did at least display an admirable spirit in the second period and could even have snatched a point, with Robin van Persie, Angel Di Maria and Fellaini all going close.
After the game, Van Gaal spoke of his anger at Smalling and said the defender was guilty of failing to control his aggression, while hinting he had had words with the 24-year-old.
"In a derby you have to be careful – the second yellow card is a stupid yellow card," Van Gaal told BBC Sport.
"The sending off is not one of those things. As a player you have to control your aggression. I didn't see the first yellow but with the second, you know you already have a yellow, so have to handle it differently. I said that to the players."
The result means United have picked up just 13 points from the first 10 Premier League games of the season, but Van Gaal is convinced his side are just as good as City and Chelsea after facing both in the space of seven days.
"We are very close. You saw it against Chelsea and now against Manchester City. They are the two best clubs in this league and the difference is zero I think."
Next up for United is a clash with Crystal Palace at Old Trafford on Saturday as they look to close the four-point gap between themselves and the top four.