"begin by saying" is correct and usable in written English.
You can use it to introduce a quote or comment that someone has said. For example: The Prime Minister began by saying, "I am proud to announce..."
Begin by saying his or her name.
Let me begin by saying sorry to Richard Ashcroft.
"Let me begin by saying something you don't hear often enough," Mr. Bloomberg said.
Let me begin by saying this book is not only timely, it's also overdue.
Callers often begin by saying, "I'm watching NY1" or "I'm watching News 12".
So let me begin by saying that tuna salad is nothing if not a brilliant idea.
Lets begin by saying that the mirror covers all of the wall.
Begin by saying something like, "We are seriously considering Mary to be a project manager here.
Let me begin by saying one thing on Palin's side of the question.
"Let's begin by saying that we are living through a very dangerous time".
Let me begin by saying: reports of the death of the book are wildly exaggerated.
