Search Results: turn down a co-workers

We have chemistry and tend to finish each other's sentences. Here are some signs but not sure if i'm reading into things too much:

1. he chooses to sit at desks near me/next to me/in my line of sight

2. on days where he doesn't sit with me as he has to sit with his team across the floor, he stands up and looks at me/for me from there (and comes over to my side to say hi and chat before he leaves office)

3. We mirror each other when sitting near each other

4. He once asked me/joked if he could join my team

5. we both stay late in office, and it feels we're now secretly coordinating when we leave -e. g. I'll pretend to work until i see him pack up, and he does the same... if i pack up first he suddenly has to leave as well. If he packs up first he asks if he should wait for me and we usually end up leaving office together almost everyday

6. he looks at my waist/down at my legs sometimes while I'm talking to him and maintaining eye contact with him

7. he keeps coming over to ask my opinion on things

8. when saying goodbye, he initiates a high/low/side five. But yesterday, he continued to grasp my hand for a few seconds and smiled at me during it, before he 'left'. He came back seconds later to ask me another opinion and we ended up leaving office together

9. He NEVER mentions his girlfriend to me. I only know of her through other co-workers. When i ask him his weekend plans, or a holiday he just came back from (i knew he went with his girlfriend), he talks as if it's just him alone doing things

10. sometimes he gets quite fidgety while talking to me

11. he tries to find out my interests. When he found out i live in a certain neighbourhood where he hangs out a lot (e. g. let's call it 'oxford'), he said 'oh so we have oxford in common? I need friends in Oxford..'

12. He's super busy but went out of his way to teach me what he does (for an hour! i didn't even ask), and turned it into a playful role-play like 'ok now your turn to do the same, i'm ready for my lesson'

This guy I used to work with is suddenly liking my pics on Instagram and then he asked a mutual friend (let's call him J, who I speak to and meet quite often) if we can all have a 'catch up'.
So J messages me saying we should all meet up.
I've spoken to the other guy probably twice in two years and I dont feel like meeting them.
They also want to meet on a day which I can't even make, so I gave another day and they say they both can't make it (without giving any reason)
Do I just go or turn it down?