Will a Greek key rug make a small room feel busy?

Not if scaled right. Choose a narrow Greek key border and low-contrast colors; a light field with gray meander keeps the perimeter defined without overpowering a 10×12 ft room.