DX8 works fine with GMod, the game looks rather dim and murky with DX8, but works relatively fine. it will disable all bumpmaps and other shaders when using it in DX8, and most postprocessing filters will not work, but you will be able to have your usual GMod experience in DX8.
i use DX8 in all source games because my GPU hates phong shading, and i’ve had no real problems so far. there are some minor kinks (such as certain models looking bad without bumpmaps/shaders) but the game is still very, very playable.
two console commands you might want to use to get a ginormous performance increase on DX8 are:
these will make distant 3D details rather blurry, but personally they allow me to use the “very high” texture setting with a constant 30fps on gm_bigcity. to revert this just type mat_viewportscale 1.
the GUI will NOT be blurry/small with this command (as running on a smaller res would), as it only upscales 3d aspects.