I like the normal scrollbars. So when I installed Ubuntu 11.04, I didn't like the new type of scrollbars. If you want to remove it:
$ sudo apt-get remove overlay-scrollbar
If this doesn't work, try this (tips from Joonas)
$ sudo apt-get remove liboverlay-scrollbar-0.1-0
Or if you just want to disable it (I think it is betters), do this:
$ sudo echo "export LIBOVERLAY_SCROLLBAR=0" > /etc/X11/Xsession.d/80overlayscrollbars
Reboot your system.
That is it!