Note: Below explained solution is not a solution of mine but rather given advice from a colleague.

1. git mv <source>.java <destination>.kt
2. Cut and paste whole content (java code) of the .kt file to different place (E.g. Sublime, Atom, or any other editor)
3. Copy content from the new place (where you just pasted you java code)
4. Paste to .kt file from clipboard and you should see below dialog. Just click yes and Android Studio will automatically convert your Java code to Kotlin. Most of the time it works fine but there are times when it can fail. But you could just simply fix those errors manually if that is the case
5. git commit -am"Converted <source>.java to <destination>.kt " (You can put whatever git message you would like to put)
6. git push origin
7. Done. Now when you create pull request Java code will look like entirely deleted file and new Kotlin code will look like newly create/added. Then you can compare two files side by side.

I would like to know if someone has figured out even better solution for this. Please comment below for your suggestions/ideas