When viewing your contacts in List View we restrict the number of records that are returned to 2,500.
The reason for this is that if you have more than 2,500 records then it would be slow to render all of the results on screen of thousands of records.
The search options give you the ability to search for contacts that you are looking for as shown below where you can search by name/email/address etc:
