Welcome to the community Roopesh. I am assuming that you are asking for content validation for SDL Tridion RTF Fields.
If this is related to UGC, you can refer the SDL live documentation.. You can handle the failure in your custom implementation.
I would firstly recommend that you think about your use case to avoid editors and publishers entering content that are not valid. By failing publishing, the users are forced to open the component, edit it and publish it again.
Ideally, you would not want to allow your users save the content, in the first place. For that, you can still refer to Nuno's blog. Using event system will help you throw custom error messages to the user. Using XML validation, you can implement it only for certain schemas.
Hope it helps!
P.S: If you would still like to fail publishing, you can still hook up to event system to use something like the below
EventSystem.Subscribe<PublishTransaction, SaveEventArgs>(YOURIMPLEMENTATATION, EventPhases.Initiated);
or include a TBB to perform the custom validation for you and throw an error. The error will be available when you click the failed publish transaction.
In any case, I strongly recommend you to validate the content on save since it provides a better user experience.