This large house was built in 1814 by Samuel Hildreth, one of Pittsford's early entrepreneurs and owner of a stage coach network serving a large part of western New York. The house was originally designed in the Federal style using timberframe construction and a cellar kitchen. It is one of the oldest buildings remaining in the village.

In the 1860s, the house and farm were purchased by Jarvis Lord, one of Pittsford's most prominent nineteenth-century citizens. Lord was a farmer, general contractor, influential politician and a successful investor with many business interests in Rochester. After he puchased the house and farm in the 1860s, he developed the property into Pittsford's first country estate. The house was enlarged and remodeled in the Italianate style with brackets, an elaborate wrap-around porch, and tall floor-to-ceiling windows. The property was improved with new barns, a circular drive, a stone wall with cast iron fence, fountain, and statuary.

In 1888 the property was bought by businessman Frank Hawley and his wife Estelle. The Hawleys named the property "Pittsford Farms" because it was made up of three smaller farms joined as one operation. The Hawleys further improved the property acquiring additional land and investing nearly $250,000 in barns and facilities for Jersey cattle and other livestock. After Frank Hawley mysteriously disappeared in 1898, Estelle began selling Shetland ponies bred on the farm. The farm gainied an international reputation for the quality of its breeding stock. The Hawleys also began the Pittsford Farms Dairy selling milk in glass bottles. Today, the dairy is Pittsford's oldest operating business.

In 1946, the house and 200-acre farm were sold to the Zornow family and remains in the ownership of the family today. Starting in the 1940s, Ted Zornow established highly successful standardbred horse and Guernsey dairy cow breeding operations. Even with a devastating barn fire in 1970, the family held on to the farm through Pittsford's period of suburban growth. To adapt to changing economics, the farm operation shifted to producing sweet corn, pumpkins and other cash crops. In 1996, the farm was listed on the National Register of Historic Places. In 1998, the development rights for the famland were purchased by the Town of Pittsford ensuring the scenic rolling fields and pastures bordering the northeast corner of the village will be protected for future generations to enjoy.