My simple definition: Web hosting is a place where the files that make up your website are kept.
Essentially this is a computer, but unlike your computer, this one is special in that it is dedicated in purpose. It is set up to be accessed by the world by allowing thousands of connections at a time so people can connect to your website.
Things are a bit different for a server. Like your home or work computer, hosting can be offered with different operating systems.
Windows is popular on work computers, but that doesn't make it the right choice for a web server. Unix and Linux are common in web servers too. Mac or Apple isn't an option.
So what do you pick?
There is a ton of info on the web about the proper server configuration for this or that, but if you want to know the most basic common choice that will run solidly for most everyone and most everything, my answer is Linux Apache servers. This is based on ease of configuration as well as commonality.
Hosting type (Shared, VPS, and Dedicated)
For the sake of this article we are discussing commonly offered "Shared" web hosting. This is different form dedicated or VPS servers. For the most part, if you don't know what a Dedicated or VPS server is, then you don't need to know.
"Shared" server hosting is what you will be offered by most hosting companies, and costs will range from $2-$20 per month.
The reason it called "shared" is because your website will be on the same server as a whole bunch of other websites. Your email will run on the same server too. Don't worry, all of your files and emails will be separated, but you will share the same server IP address.
The fact that you are sharing a server address means that activity coming from someone else on your server can get that server address blocked or blacklisted by others on the web if it does things like send spam emails. There is risk here in that you don't know what others are doing, but good hosting companies monitor activity on servers and have policies in place to make sure others you "share" with will behave.
Bandwidth, Storage, Databases and other details
Frankly, These specs vary all over the place. For most people running a typical website like Wordpress, Joomla, Drupal, or any HTML/PHP based site. you should find hosting that offers at least 1 SQL database. All of the memory and bandwidth specs depend almost entirely on how big your site is and how much traffic it will receive. That is tough to figure out, so I recommend starting with a "better" plan.
What do I mean by a "Better" plan? Most hosting companies offer 3 choices in shared web hosting. The names change but the marketing approach is the same. Its called "Good, Better, best".
Basically the marketing department figures out what most people need, and offers a mid level plan ant a good price point (this is the "Better" plan). The cheap plan ("Good") is designed just to be cheap and usually is also designed to be missing some features most people need, and the top most expensive plan ("Best") is typically more than what most people need.
fact is, almost every company will allow you to upgrade at any point should your "better" plan not meet your needs.
What about the Domain name isn't that part of it?
This is a common point of confusion. Many companies offer both services. Domain registration is a completely separate service that web Hosting. You may, in fact, register a domain name and not direct it to any website or web hosting at all. You may also have web hosting but not have a domain name assigned to it yet. You may have a domain name at "Company A" and a web hosting account at "Company B".
Benefit of getting your domain at the same place as your web hosting. Typically when you buy a domain and hosting together the company you buy from delivers your domain already pointed to the web hosting account. This means you will not need to access your domain name separately and set the DNS records yourself. Read What is a domain name.