King wanted to bring Maa Tarini along with Lord Madanmohan , Raghunathjew to Puri. They prayed them to bring them. Thus they came to Kalinga or Odisha. After that Sri Purusottamdev was pleased and wanted to reward Sri Gobinda Bhanja. Sri Gobinda Bhanja asked to take the deity “MAA TARINI” along with him to Keonjhar state. King Purusottam Dev agreed to this and asked him to ask Maa’s permission. Govinda Bhanja prayed MAA and MAA agreed with one condition. Maa said she will go with him as long as Govinda Bhanja doesn’t look back to see Maa. With this Condition Maa went with Govinda Bhanjha to Kendujhar(Keonjhar). On the way from Puri to Kendujhar, Maa horse hoof sounds were audible. On the forest of Kendujhar, Maa’s horse hoofs was not audible to Govinda Bhanja. He got scared that Maa stayed behind and he turned to look back to see. There n then Maa turned into stone at Ghatgaon at the present place where the deity is installed. Govinda Bhanjha prayed Maa for forgiveness but Maa said she has made the condition with him before coming. And she will stay in the wild jungle and asked Govinda Bhanjha to worship in that place. Till today Maa Tarini is being worshipped in Ghatgaon. Which is now a major tourist attraction in Keonjhar (Odisha).

In the year 1970 the management of the deity was took over by the Orissa Hindu Religious Endowment Department of Govt. of Orissa by continuing a Board of Non-hereditary-Trustees under the provision of O.H.R.E. Act. The Present temple was constructed in the year 1980 by the then Trust Board.