Srila Bhaktivinoda Thakura explains that the word adi-vasya refers to one who has been living with another for a very long time. Govinda was addressed as adi-vasya because he had been living with Sri Caitanya Mahaprabhu for a very long time, whereas other devotees, who were mostly new, would come and go. In effect, the Lord told Govinda, "Since you have been living with Me for a long time, you should not be foolishly disappointed in this situation. Bring all the food to Me, and you will see that I can eat it."