If the caliper moves more than a fraction of an inch something is missing. Make sure the brake pads are in the caliper, both pads. You may need to dismantle the caliper and make sure the piston is not binding.