To tag or not to tag—when author feedback gets personal

Would you walk up to an author at a book event, or on the street, and tell them how much you hated their book? No? But by creating a public work of art, surely authors are opening themselves up to scrutiny and should be prepared to take the rough with the smooth?