Is there any atoms in ice cubes?

think so

Water molecule contains one oxygen and two hydrogen atoms.
The only thing that makes solid water (ice) different from melted water is the arrangement of molecules.