Black Sabbath’s first manager says Ozzy Osbourne’s bandmates belittled him for a perceived lack of talent during an initial struggle to develop his singing voice.

Jim Simpson oversaw the group’s early years – including the release of their first two albums and breakthrough single “Paranoid” – before being ditched in favor of legendary mogul Don Arden. He revisited some hometown landmarks from Sabbath history during an interview with the Birmingham World to mark Osbourne’s 75th birthday on Dec. 3, showing off the first-ever group photo from 1968.

“The Ozzy I see now on film – it’s hard to relate him to the Ozzy I knew well,” Simpson admits. “Ozzy was a simple, straightforward guy – and by simple, I don’t mean stupid. I mean uncomplicated. He was fun to be with.”

READ MORE: How Black Sabbath’s Debut Gave Birth to Heavy Metal

Back then, Simpson said Osbourne “was never very confident in what he could do. All the other guys could play an instrument and pick it up and make a noise. He wasn’t a trained singer. He just decided to be a singer, and he was never quite convinced he could get away with it.

“So he used to come round to my house and listen to these old blues records with singers like Jimmy Rushing and Jimmy Witherspoon, and he got interested in their backgrounds,” Simpson added. “Ozzy was always thirsting for information, he was a very open and trusting kid … asking questions all the time about the origins of music. He liked learning and he’s a very good learner.”

Simpson added that Osbourne took his cue from some of those old singers to “shout above” Black Sabbath’s music: “Like Jimmy Rushing, he had this voice that started somewhere in his stomach and it came out, in the nicest possible way, as a bellow. It was a big voice … but he was very much lacking in confidence and needed to be bolstered all the time and told he could do it.

Young Ozzy Osbourne’s Parents Had to Co-Sign His First Contract

Meanwhile, “the other guys in the band rather sneered at him, looked down at him a little bit because he couldn’t play an instrument,” Simpson said. “As he grew up, he became the main man in Sabbath and, after a while, everyone began to see that.”

Simpson, who remains active in the Birmingham music scene, also remembered having Osbourne’s parents co-sign his management contract since he was too young to do it alone. “His dad was very interested in how we were going to make Ozzy famous,” Simpson said. “They were very civilized if rather bewildered by it all, and confused as to why their John was going to be a rock ’n’ roll star as we all believed. But they got used to it, I think.”

As for the moment when Black Sabbath left him behind, Simpson said they “all walked out on their contracts after a couple of years – but at least we had something to go to court with.”

Watch Jim Simpson Discuss Black Sabbath’s Early Days

Think You Know Ozzy Osbourne?