I have often found myself in difficult situations without knowing what exactly is the difference between host and hostname

/etc/hostname -> Assigns a name to local machine and you can set the name of the machine by editing this file. It can be a FQDN eg. vps.example.com. You will also need to edit /etc/sysconfig/network to specify the hostname in Centos, if /etc/hostname is missing etc.

/etc/hosts -> Maps hostnames to IP addresses locally. It has nothing to do with DNS or FQDN. You have to specify hostname and IP address for resolving on line by line.

67.1.2.3.5 vps.example.com  vps vps

Imagine this, you are on a server disconnected from the internet and when you type vps.example.com on your browser, it resolves to the IP you specified on the /etc/hosts file.