I own the 36/silver/fluted/jubilee. I love it. There's something about the monochromatic look in the 36mm size that I really enjoy. It really pops and I routinely get compliments on it.
One man's opinion but in the 36 I prefer the silver to any of the other dial colors. The monochromatic look kind of strikes me as a poor man's YG DD 36 with champagne dial - and I absolutely mean that as a compliment.
If I were buying a DJ 41, I would feel oppositely. To me the larger size is an inherently "sportier" watch than the 36 and thus I would be more inclined to go for one of the colored dials - probably choosing the blue.
To close I will say that I have not found my silver 36 to be too "dressy" at all, and I am a very casual dresser. I'd have no concerns about wearing it anywhere from the beach to the office. However again, opinions may vary. For reference my wrist is between 7-7.25 inches.
Edited to add one last thing: the blue is absolutely going to be a more legible watch than the silver is so if that is one of your primary considerations, give it the weight it deserves. Personally I find the legibility of the silver to be perfectly fine outdoors but with my aging eyes it can suffer indoors when in dimmer lighting.