Our conception of time is just totally wrong, or at least this is true for 99.9 % of all humans and that is certainly an underestimate. Something like THE time does not exist, we are just all living together in a small bubble in the universe, all of us experiencing more or less the same gravitational field, all of us moving together at more or less the same speed as Earth moves through space, and that is what makes it look like there is THE time. But actually time is more like counting the calories you burn as you walk through the world, it is specific to you and the path you take. There is a great lecture »The Physics and Philosophy of Time« by Carlo Rovelli [1] aimed at the general public that does a good job pointing out how wrong we are.
Perhaps: the field of physics, in its discussions, allows for local observation and within local observation, the observation of local causal effects. The lack of causal effects in your observable context when a distant event occurs means that the time of the event to your local context is the time of the effects of that arrival (when you actually have evidence of it). One might suggest this is a correlate of the "no" answer to the "if a treefalls in the woods but no one is around to observe it, does it make a sound?" question but... On the more practical side you can consider CPU clocks and time synchronization. One can only synchronize two clocks to agree up to a time span but within that span the relative ordering of events at each is undefined. The physics reason is that only causality establishes order, the cause precedes the consequence and beyond causal entanglements, ordering is undefined.
I'm not a physicist but I've been pondering this sometime...
Two cars, both moving forwards, can still move in different directions! So does "forwards" exist? Not as a global direction, but once you have picked an orientation it is a useful label for a distinguished axis.
Time is like that. Pick a velocity, and then time is a distinguished axis in the coordinate system you overlay on spacetime.
But it doesn't make sense. OP is saying that there is no absolute measure of time, it depends on the frame of reference of the observer. Therefore there is also no absolute measure of speed.
Correct. A fly on the seat of your car travels 0 m/s with respect to the car and to you, but travels, say, 30 m/s with respect to the road. Speed (and velocity, for that matter) always needs a reference to be meaningful.
[1] https://www.youtube.com/watch?v=-6rWqJhDv7M