In exploring the physics and geometry of the universe, Stephen Hawking became a world-renowned pioneer of black hole theory, writing the bestselling book A Brief History of Time, which has sold more than 13m copies, and inspiring people to “look up at the stars and not down at your feet”.

But, during Hawking’s student years and as he approached adulthood, his father was deeply concerned about how his son would turn out. Frank Hawking lamented that “he hangs round the house with little initiative and does not study much”, according to previously unknown diaries that he had written partly in code.

Those diaries are among family papers and photographs to which a Costa award-winning biographer and physicist has been given unprecedented access.

In September, Graham Farmelo will publish the first definitive biography authorised by the Stephen Hawking estate, the publisher John Murray will announce this week. As part of his research, Farmelo has been shown previously unknown material ranging from the diaries of Hawking’s father to the letters and journals of his mother, Isobel. They had been kept until now in the home of Hawking’s sister Mary.

Farmelo said: “It was a wonderful, completely unexpected bonus to be given access to these diaries and papers. They are a 24-carat source of information about Stephen Hawking’s life, especially his formative years and the harrowing months after his diagnosis of motor neurone disease when he was only 21 years old.”

He said it offered a “raw and honest insight” into Hawking’s upbringing and the devastating diagnosis in 1963 of a fatal degenerative disease, which was to leave him almost completely paralysed.

Hawking defied medical expectations that he would die within two years. He died in 2018, at the age of 76, having proved himself as one of the most celebrated minds of our time, carrying out groundbreaking work in cosmology and theoretical physics, exploring the mysteries of space, time and black holes.

His achievements were all the more inspirational after he started using a wheelchair and was only able to communicate through a computer and a voice synthesiser. “Life would be tragic if it weren’t funny,” as Hawking put it. “My expectations were reduced to zero when I was 21. Everything since then has been a bonus.”

He said: “Remember to look up at the stars and not down at your feet. Try to make sense of what you see and wonder about what makes the universe exist. Be curious. And however difficult life may seem, there is always something you can do and succeed at. It matters that you don’t just give up.”

But, in 1961, Hawking’s father, an expert in tropical diseases, could never have imagined his son’s later achievements when he wrote in his diary: “We are a little worried at the way Stephen is turning out. He hangs round the house with little initiative and does not study much.

“[Isobel] says he has an inferiority complex to me (he has no need to) and he has lost faith in physics at Oxford, thinking it is inferior to arts. This is a great pity if so. At his age I had a burning ambition to get on, and if only I had had half his advantages, I should have done much better.”

Frank kept a diary for more than 60 years and wrote many entries in a secret code that Farmelo has managed to crack – translating more than 200,000 words relating to his son’s childhood, his illness, his two marriages and his career as a world-class physicist.

Frank wrote: “This journal was originally written in Greek script to form a simple secret code and so secure greater privacy, which is essential for a journal which may fall into the hands of enemies or easily wounded intimates. Since the Greek alphabet does not contain all of the English letters, the following adaptations have been made …” He added a code for the letters H, V, QU, W and J.

He struggled to come to terms with his son’s failing health, the diaries reveal. In 1967, he wrote: “I find it a slow and ghastly experience with [Stephen]. Everything is so dreadfully slow and long drawn out. And his speech is so slow and difficult to understand that conversation is very difficult. I am very sorry for him and will do all I can for him. But I don’t enjoy being with him.”

Farmelo’s critically acclaimed book The Strangest Man: The Hidden Life of Paul Dirac, Quantum Genius, won the Costa biography prize and LA Times book prize in 2009. The British physicist Dirac was a personal hero of Hawking’s.

For his latest book, Farmelo has interviewed the closest members of Hawking’s family, including his sisters, Mary and Philippa, first wife, Jane, and three children, Robert, Lucy and Tim.

The biography, titled Hawking, will be published on 24 September by John Murray, which describes it as “the definitive portrait of an exceptional life and intellect”.