I want to make the custom payement gateway in the shopify so which programming language would be suitable for making plugin and also give some hint how to proceed in it.
Since Shopify apps integrate via REST any app language that you are comfortable with will work. Shopify officially supports Ruby, Python and Node
However as @david-lazar writes you cannot directly integrate a gateway. Shopify does support dozens of payment processors so you really should look at all of those to see if one meets your needs.
If you really need your own payment processor you'll have to either
This will let you integrate your payment gateway but you'll also have to write the whole checkout. The Sales Channel app would have to go through PCI compliance certification (unless you want to use it with Stripe) and your liability insurance would probably require your custom checkout to go through it as well.
A Sales Channel would require the whole theme to be adjusted. You have to adjust: the add to cart; cart display; and checkout. The custom app would require all your checkout links to be adjusted so that the checkout process ends up at your app instead of at the standard checkout.
It's all quite doable but unless you have a compelling business reason (e.g. scheduled delivery or your own custom discount scheme) then you are way ahead finding one of the supported processors that will work with your business.