Please read ISO-32000-1 (this is the ISO standard for PDF), more specifically Table 30 ("Entries in a page object"). It defines the
Rotate entry like this (literal copy/paste):
The number of degrees by which the page shall be rotated clockwise when displayed or printed. The value shall be a multiple of 90. Default value: 0.
Whenever an ISO standard uses the word shall, you are confronted a normative rule (as opposed to when the standard uses the word should in which case you're facing a recommendation).
In short: you are asking something that is explicitly forbidden by the PDF specification. Meeting your requirement is impossible in PDF. Your page can have an orientation of 0, 90, 180 or 270 degrees. You will have to rotate the contents on the page instead of rotating the page.
Click this link if you want to see how to answer this question in iText 5.