Joe O’Brien was born in Shettleston, Lanarkshire, in 1875. He played football for Baillieston Thistle, Clitheroe and Sheffield before signing for Southern League club Reading in 1896. He earned a “big reputation” with Reading, was selected for the Southern League representative XI to face the London Association in 1899, and signed for Football League First Division club Blackburn Rovers in April 1900.
In the absence of Allan Hardy, he went straight into the team for the league match against Sheffield United on 13 April, which finished as a 3–3 draw. He made two more league appearances before returning to Scotland where he spent the 1901–02 season with Aberdeen, for whom he scored twice from eleven Northern League matches.
For his next club, Swindon Town, he played in all five FA Cup ties and all but one Southern League fixture in 1902–03. He spent the folliwng season back with Reading, before joining Brighton & Hove Albion another Southern League club, for 1904–05. He was a regular in their side, playing 33 matches and partnering Tom Robertson at full back, before returning to Swindon Town for 1905–06.
This time he was less of a first-team regular, appearing in 21 Southern League matches and 14 in the United League, a secondary competition. He finished his football career in the north-west of England with Stalybridge Rovers and Haslingden.