The answer to your question depends on three parameters:
The PDF version: before PDF 1.5 vs. PDF 1.5 and higher,
the PDF style: plain text cross-reference table vs cross reference stream, and
the iText(Sharp) version: before 5.3 vs 5.3 and higher).
Let's start with the PDF version and the cross-reference table. In case you didn't know: the cross-reference table defines the byte offsets of every object inside the PDF.
In PDF versions prior to PDF 1.5, the Cross-reference table is added in plain text, and each byte offset is defined using ten digits. Hence the size of a file will be limited to 10 to the tenth bytes (approximately 10 gigabytes).
The maximum size of a PDF with a version older than PDF 1.5 is about 10 gigabytes.
Starting with PDF 1.5, you can choose whether you want to create a cross-reference table in plain text, or whether you want to use a cross-reference stream. If you use a cross-reference stream, the "10 digits only" limitation disappears.
The maximum size of a PDF with a version PDF 1.5 or higher using a compressed cross-reference stream depends only on the limitations of the software processing the PDF.
If you use iText(Sharp), you need to make sure that you create your files with a compressed cross-reference stream if you need files bigger than 10 gigabytes.
The version of iText also matters:
All versions prior to iText 5.3 only supported files up to about 2 gigabytes because all byte offsets are stored as
intvalues in those versions.
From 5.3 on, iText supports files up to 1 terabyte because all byte offsets are stored as
longvalues in those versions.
The maximum size of a PDF created with iText versions before 5.3 is 2 gigabytes. The maximum size of a PDF created with iText versions 5.3 and higher is 1 terabyte.
However: you also need to take into consideration that not all viewers can render a file that big. While the file may be completely compliant with ISO-32000-1, you may hit memory limits.
You are sending 1 gigabyte over the internet to a random PDF viewer plugin in a random browser on a random machine. If it's an old machine, it won't even have 1 gigabyte of memory. When the end user has a slow connection, the end user will have to wait forever to get the full file. There is no way you can predict how the browser and the PDF viewer will respond. My guess is that the browser shuts down because of an out-of-memory exception. This is not a problem that is caused by a limitation that is inherent to PDF, nor to a limitation of iText(Sharp). It is purely a limitation that is inherent to your design. You shouldn't send gigabytes to a browser. Write them to a shared drive in the cloud instead.
Click this link if you want to see how to answer this question in iText 5.