While no official information has been provided on their nationalities, local media say they are of African descent.
Rescue teams are searching the area for more victims of the sunken boat, which was travelling to Lesbos from Turkey.
On Tuesday, the Greek Coast Guard, accompanied by a European Union patrol boat, said it rescued dozens of people from the Aegean sea after their vessel sank at about 07:00 local time (04:00 GMT).
Several boats and a helicopter are continuing to search the stretch of water between Lesbos and Turkey, while survivors are being questioned about the number of passengers on board, the Greek newspaper Ekathimerini reports.
Under the deal, anyone who arrives on Greek islands must be returned to Turkey unless they qualify for asylum.
In recent months, the campaign group Human Rights Watch (HRW) has reported allegations of violence at the Greece-Turkey land border as refugees and migrants are “pushed back” into Turkey by the Greek authorities.
Media captionThe refugee families caught up in a war zone
On Sunday, the UN refugee agency UNHCR warned that a lack of rescue ships and lifeboats coupled with an increase in migrant vessels leaving Libya also raises the risk of a dramatic rise in fatalities in the Mediterranean Sea.
“If we do not intervene soon, there will be a sea of blood,” spokeswoman Carlotta Sami said in a statement published in the UK’s Guardian newspaper.
Libya, which has been torn by violence and political instability since long-time ruler Muammar Gaddafi was deposed and killed in 2011, is a key departure point.