You should not rely on this information to make or refrain from making any decisions. Want More? Charly Lester talks how to tackle the second date. You have no way of knowing. To answer that question, I brought in the big guns: Guys can often do this in very subtle ways.
Just to give you the range of perspective AskMe provides, OP, I would have a strong instinctive recoil if a guy I had just met moments before started casually touching me while we were out for coffee or something.
I have also had trouble getting those second dates. At least be a respectful human being and be straight up with how you feel.