Martin Smith, from Drogheda in Co Louth, was on holiday in Praia Da Luz with his family when they bumped into the man just before 10pm, May 3. The Smith family's suspicions were aroused because the man made no response when they asked if the barefoot child was asleep.
"He just put his head down and averted his eyes, which is very unusual in a tourist town at such a quiet time of the year," said Mr Smith.
Initially the Smith family thought nothing more of the encounter - and even the next day when the story broke they still didn't make the connection.
"We were home two weeks when my son rang me up and asked was he dreaming or did we meet a man carrying a child the night Madeleine was taken," said Mr Smith.
"We all remembered the same recollection, and I felt we should report it to the police. We've all been beating ourselves up that we should have made the link sooner, if only we'd remembered the next day.