Limited access is a permission level that is automatically assigned by SharePoint, at a container level, to an account that has access to an entity within the container. For example, a site’s permissions declare UserA as having Limited Access when UserA has been granted Contribute access on a library (that has broken permission inheritance) inside the site. Likewise, a library’s permissions declare UserB as having Limited Access when UserB has been granted Approve access on a folder (with broken permission inheritance) inside the library.
Limited Access is necessary but it can be annoying. Where Limited Access is granted to a user/group on a list or a library, the user/group can actually navigate to the entity without getting Access Denied. The list or library just appears empty. So this means, if I want to open a single library in my site to the whole company (like a dropbox), everyone will be able to see and navigate to all of the other lists and libraries in the site as well, but they would appear empty. This is because Limited Access is auto-assigned at the site level and therefore all lists and libraries inheriting from the site have the access level also. There is absolutely nothing you can do about this. Removing the Limited Access declaration from the site will remove the lower level access as well.
A work-around, to stop users from seeing other lists and libraries in the site, is to use Audience Targetting in the Navigation settings. This doesn’t prevent users from navigating to entities but it does hide the link in the Quick Launch. I am not recommending this though as it would require extra maintenance but it is an option.