The Iranian team, who had lost to Qatari club 5-0 three years ago, defeated the team 3-0 at the Tani Bin Jassin Stadium in Group D.
Gholamreza Rezaei scored an early goal just two minutes into the match. Three minutes later Eamon Zayed was on target.
Al Gharafa dominated the match afterwards but failed to score in the first half.
Javad Kazemian, who came off the bench in the second half, scored the visiting team’s third goal in the 72nd minute.
Al Hilal of Saudi Arabia was held to a 1-1 draw by Al Shabab of the UAE in another match of the group.
Persepolis is the first team with seven points followed by Al Hilal with three points.