Baysgarth’s full origins are unknown. Most of the South Front of the house dates to around 1740; the East Wing, however, may date back to the 17th century, with additions being added as late as 1890. The house as it stands today possibly replaced an earlier timber-framed building.

The first owner was Thomas Glentham.  However, the best known owners were the Nelthorpe family, who owned the house between 1620 and 1792.

Following the Nelthorpe’s ownership the house changed hands frequently. In 1889 it was sold to Robert Wright Taylor, whose family grew up in it throughout the pre-WW1 period. The house was later given to the Public in 1930.