In germany you need to differ between "company" and "freelancing". The latter one is a bit more difficult to get, you need a university degree at least. Tax offices in germany are very picky. If you run a company, you need to pay IHK fees and a special tax (gewerbesteuer).
You also should know that not all your services can be invoiced as freelancer. For example web design is billed via your company.
That being said, in case of a company you need to get a Gewerbeschein from your local city. In case of freelancing work only you simply need to tell your tax office. They will send you some forms you have to fill.
Before you start this, you should read about the Gründungszuschuss, a financial help from the government:
Its a good bunch of money, you should really read about it.
After all, paying taxes is pretty easy. You will need to download Elster, the software for taxes and then pay each months your Mehrwertsteuer. If you have done it one time, you can do it all the time. At the end of the year you make an Einkommenssteuer and thats it.
I would recommend you some good software (Wiso Steuersparbuch) and to show your tax forms to a friended tax expert. Then you are safe.
Writing invoices is pretty easy. You can read some links in the internet about it:
Basically your Umsatzsteuer ID must be there, your full name and contact, the included Mehrwertsteuer and a date.
I am very sorry that I can't give you a link which contains all the information on one page. I could have that aswell back in 1998. Anyway, I hope my little text gives you a good start.
PS: non-german speakers: sorry for the german terms, but I could not translate all in matching english words