An employee said he and other staff did not go to work on Tuesday because of the Shabaab threat.
"Their militants stormed our offices in Wajid and Baidoa and took control of them. All staff members were told to leave the offices, including the security personnel," the employee said, on condition of anonymity.
The World Vision staff member said its officials were due to meet al-Shabaab leaders in Baidoa on Tuesday to discuss the future of their operations.
According to the NGO's website, the organisation has been active in Somalia since 1992 and has been focusing on primary health care in the Wajid and Baidoa regions.
Adan Dhubow, a Baidoa resident, said al-Shabaab gunmen were guarding a World Vision tuberculosis centre, allowing patients to pick up their medicine but denying access to everybody else.
Al-Shabaab officials contacted by AFP declined to comment.
The Shabaab, which have declared their allegiance to Osama bin Laden's al-Qaeda network and control most of southern and central Somalia, have imposed strict conditions on foreign humanitarian organisations.