The son of the Blackburn Rovers and England winger William Townley, Jimmy Townley was born in Blackburn but spent most of his youth in Germany where his father held a coaching post. After playing for Hamburg Victoria and the Swiss club St Gallen, Jimmy returned to England in 1924 to try his hand in the Football League, firstly with Chelsea where he had an unfruitful trial, and then with Tottenham Hotspur for whom he signed professional forms in March 1925.
His progress at White Hart Lane was hindered by a broken leg, however, and he made only three First Division appearances in just over three seasons.
In August 1928, Jimmy joined the Albion for a fee of £600, an enormous amount for the club at the time, but he failed to make an impression. Played at inside-left in the opening seven games of the 1928-29 season, he was dropped on Jimmy Hopkins’ return from injury, and, after appearing at centre-forward in a couple of games later in the campaign, returned to the reserves before transferring to Clapton Orient In February 1930.
After nineteen League outings with the Third Division (South) side, Jimmy was released in May 1931 and returned to Switzerland.