Charlie Baker was born in Turners Hill, Sussex in 1936. He played local football while completing an apprenticeship as a toolmaker and his National Service with the RAF before joining Brighton & Hove Albion in 1960.
He made 81 league appearances over four seasons, which included 18 months as regular first choice goalkeeper, but remained a part-time player while working for an engineering company. He then played a further 28 league matches for Aldershot before returning to non-league football with Crawley Town.