Closing a web part by selecting Edit -> Close will remove the web part from view but the web part and it’s settings are maintained in the page. If you then access the Advanced Web Part gallery and options panel (by clicking Add a web part and selecting the link in the bottom right of the Web Part gallery list) you can re-add the web part to any web part zone by clicking Closed Web Parts and dragging the web part onto the page or selecting a zone and clicking Add. The Web Part will be restored with the settings that were defined before it its closure.
Deleting a web part by selecting Edit -> Delete will permanently remove the web part from the page. After re-adding the Web part from the Web Part Gallery, the Web Part will exhibit the default settings.
A handy trick is to append ?contents=1 to a target page URL to access the Web Part maintenance page. This is useful as a quick fix when a web part is throwing and error which is preventing the page from displaying. From the maintenance page you can simply close the troublesome web part until the underlying problem is identified and resolved.