David Letterman apologized to his wife, Regina Lasko,  on The Late Show last night for having sex with female employees that ended up landing him in him in the middle of a huge scandal.

“She has been horribly hurt by my behavior,” Letterman said to his audience, “and when something happens like that, if you hurt a person and it’s your responsibility, you try to fix it. And at that point, there’s only two things that can happen: either you’re going to make some progress and get it fixed, or you’re going to fall short and perhaps not get it fixed, so let me tell you folks, I got my work cut out for me.”

But in true Dave form, he cut the seriousness with humor, saying, “Wow, that weekend went fast.”

Another audience member said, “He made a joke about loving the fall weather in New York, and about how it was getting mighty chilly. He was making fun of the situation. If it’s chilly here, you can only imagine how chilly things were at home.”