In a large saucepan, combine graham crackers crumbs, brown sugar, milk and butter. Cook over medium heat for 5 minutes, mixture should come to a boil. Stir constantly while mixture boils.
Pour over crackers and top with another layer of crackers. Melt together the chocolate chips and peanut butter in a microwaveable safe bowl. Spread the chocolate/peanut butter mixture over crackers. Refrigerate.
When completely cool, cut into bars.