To trigger your custom the event handler after the Rollback button is clicked, you'd have to get the controls of the Rollback button and register your custom function on an onclick event. In your event handler, check if the view is HistoryListDialog
and get the controls of the Rollback button using its id RollbackBtn
. Then you can register your onclick custom event on the Rollback button.

Configure your GUI extension as you'd normally do and in the constructor of the JS, register a display event handler $evt.addEventHandler($display, "start", this.getDelegate(this.onDisplay));
. And the rest of your JS should be something like this:
function onDisplay()
{
var view = $display.getView().getId();
if (view == "HistoryListDialog")
{
$display.resize();
// Get the controls of the "Rollback" button using its id.
var BtnRollback = $controls.getControl($("#RollbackBtn"), "Tridion.Controls.Button");
// Register an onclick event on the "rollback" button
$evt.addEventHandler(BtnRollback, "click", this.getDelegate(this.onBtnRollbackCustomClick_));
}
}
function onBtnRollbackCustomClick_()
{
console.log("Rollback button clicked");
}
This TREX post might help too.