You can’t go wrong with basic rope. Duct tape and saran wrap are cheap and easy options as well. Even old neckties, which you can probably get at a charity shop, are a good option. And don’t forget belts! You can also fairly easily find toy metallic handcuffs that can open with or without a key. They’re pretty cheap, but if you’re new, I wouldn’t want to spend a lot of money right away. Experiment around a bit first.
At the next level, you can invest in some basic restraints. Even most mainstream (meaning straight and vanilla) adult/porn stores sell basic bondage kits. They typically include wrist and ankle restraints, and a blindfold or collar. And if you’re new, it’s a good way to start. Combine that kit with rope, or some carabiner clips or D-clips from a hardware store, and you’re well on your way.
Still want more? You can also visit my Shopping Page on my blog (links near the top) for places I go to get stuff online. (Plus, if you buy them from my links, you help support my blog.) For the beginner, I’d recommend Extreme Restraints, because they’re good quality for the price. Mr. S is fantastic quality, but can be more expensive. Have fun exploring!