As can be seen it is a whole lot inconvenient and you have to visit websites unnecessarily. The "search by image" button has also been removed.
This comes after Getty sued Google alleging it allowed people to steal its images. Google removed the "view image" and "search by image" buttons and is displaying copyright info more prominently to settle the case.
Oh well, I guess its time to use Bing now.
PS: You can still right click to open image in a new tab in your browsers. But people have said that in some cases even if you do this it only loads Google's cache of the image not the full size image from the website itself.