Combine condensed milk, sugar, eggs, vanilla, and 1 quart milk. Heat in microwave; let cool. When cooled, add peanut butter and 1 quart milk. Mix well. Pour into freezer and freeze.