Save your images on WordPress is something that happens on its own. You do not need to do anything special or different to do that. Once you upload an image on your media gallery to upload to a post you will see that it gets automatically saved in the site. This happens because WordPress does a lot of things to save it.
There is an address on your server which is /wp-content/uploads/folder address. This is ideally where WordPress saves your images and all the media that you upload. It is also noted that once a month or so WordPress organizes all the uploads that have been done by you to keep it in a proper way and an organized way on the basis of your folders.
Yes, you can also view these folders on your own by connecting to your WordPress site using the FTP client. It is also known that WordPress saved the information of all the posts available in a post meta table whenever you are to insert any kind of image to any post or pages.
WordPress has a feature called the featured image which you can set. They are also called thumbnails. WordPress saves this type of information as a meta key_thumbnail_id address and in return stores it in the post meta table of the database.
You also have the option to delete these details from the server but if you use a FTP client then it is sure to delete it from the server but not the WordPress database. It will still be shown to the viewers but only as broken image links so it is advised to remove them from there too.
Finally, when you know the place it is saved, you can delete the references to the images and media uploads from the database, following which they won’t be shown in the Media library even though all the images will be saved on the server.