Dr. Joseph Peaslee, Jr. was born in 1646 and raised in Haverhill near Rocks Village. He married Ruth Barnard in 1671 with whom he shared a close relationship. Too close it seems, for the sensibilities of their fellow townsmen, as they had a baby six months after their wedding day and were soon thereafter, presented to the Salisbury Court for fornication, a charge to which they readily admitted. They were sentenced to a whipping or to pay a fine of six pounds. The records do not show which punishment was carried out.

In 1673 in the midst of the Indian incursions Joseph built a fortified house in East Haverhill where he prospered as a millwright and a farmer and was known locally as a physician. The Peaslee-Garrison is the oldest house in Haverhill and endures today at 790 East Broadway. Garrisons were safe houses constructed with bricks and they often had no staircase to the second floor to thwart Indian raiders. Many of the bricks used by Peaslee were imported from England. A chimney is located at each end of the house and it originally had three rooms upstairs and three rooms downstairs. The woodwork was said to be elaborate for its day and the richness of the house suggested social prominence.

Dr. Peaslee, however, had a split personality. Although he was generally a respected member of the settlement at other times townsmen considered him a nuisance or worse. He was in and out of court on numerous occasions both as a plaintiff and defendant for various misdeeds several of which reveal a darker side to his character.

In November of 1676 Peaslee was charged with assaulting one Peter Bruer with a chain. He was ordered to pay restitution to Mr. Bruer in corn and money. He was also bound to good behavior. Later in April 1677 he was convicted of abusing Timothy Swan and ordered to pay 20 shillings in corn to Timothy's father, the contentious Robert Swan.

Then in 1681 he faced major assault charges for a brutal beating of one Peter Green following a barn-raising, ordinarily a neighborly affair. The cause of the altercation is unknown but along with several other witnesses, Ephraim Roberts testified that Peasley had chucked Green under the chin, and then took an apple from a tree and stomped it to pieces boasting: "I could do with you as to this apple, and deal with you and throw you into the river." Peaslee challenged him to a fight but Green refused and began walking home. After following him for a short distance Peaslee struck his victim from behind and beat him until he lost consciousness. He told Ephraim Roberts that Green was drunk and to haul him under a log and leave him there. Peaslee was found guilty, fined and forbidden thereafter to bear any weapons.

This ornery disposition apparently took a turn for the better when he found his religious compass with the “Society of Friends.” The Peaslees were an independent clan and were among the first to declare themselves Quakers. When the community petitioned for a new meetinghouse in the East Parish, Dr. Peaslee made a formal request that the Friends be allowed to use the meetinghouse “for and in their way of worship” but his request was turned down. Instead he opened his home to the first Quaker meeting in New England. Joseph and Ruth were founders of the fledgling Friends movement in Haverhill and were ancestors to the renowned poet and Quaker John Greenleaf Whittier through their daughter, Mary’s marriage to Joseph Whittier.