What if I smile while not working out?

I was thinking about fasting today.

Not thinking about fasting in a "planning to fast, so I'm thinking about it" kind of way, just thinking about the whole idea. Jesus gave us instructions not to appear in public looking wan and peaked when fasting, just so everyone who sees you can think, "Wow, look how awful he looks! He must be fasting!", but instead to appear as you normally would. And so to avoid announcing my fasting before my coworkers, I usually manage to avoid (or downright dodge) questions about what I am doing for lunch, or what I ate, etc.

So here is where I ran into a snag in my thinking. I am physically active every day, even if that is limited to going for a walk at lunch. But I often ride my bike to/from work, or hop on the treadmill for a few minutes, or whatever I can fit into my day. On days that I fast, though, I do none of those things; my reason being, I don't want to burn off calories when I am not taking any in, and end up so hungry that at the end of the day I am focused on food, and not God. But now I'm wondering, is forgoing those things really going about my day as normal? Am I tipping off everyone around me by acting strangely on days that I fast? Am I over-thinking this?

I imagine that I am over-thinking it. But I do wonder. Any thoughts?