At the Urban Bean Coffee House in Orange Park, where Sheriff Darryl Daniels has his own special blend, both coffee and questions are brewing.

Daniels has been noticeably out of town following the revelation he had an affair with a subordinate during his time in charge of the Duval County Jail.

He reportedly later had his deputies arrest the woman for allegedly stalking him. The FDLE is reviewing to see if that was an abuse of power.

Daniels was in Washington D.C. when it all came to light, according to a department spokesperson. He returned to the First Coast Thursday night, only to leave again Friday morning for Kentucky. The spokesperson did not say why or when the sheriff would return.

Residents say they’d like to see him return as soon as possible to do the job they elected him to do.

"He needs to be in charge of all of us to make sure that we’re safe and everything’s working,” said Lois Andrada, Clay County resident.

“I think he should be here taking care of his job, making sure Clay County and Orange Park are safe,” said Paige Mata, resident.

Daniels broke his silence around 1 p.m. Friday with a written statement He said nothing about his absence or when he would return but apologized for personal decisions.

"My family has asked that I not discuss private matters in public so I will not be commenting further on the personal aspects of this issue,” Daniels said.

Mata believes the sheriff’s privacy should be respected.

“I don’t think he should have to talk about that, I don’t think any of that should matter,” Mata said.

Andrada said as an elected official, a written statement is not enough, especially given Daniels’ absence.

“With everything that’s going on with him he still needs to take accountability for his actions,” Andrada said.