It's just the tape - it can hold VERY tightly. Heat it up with a hair dryer and then use a plastic panel pry tool and it will pop off... or you may be able to just remove the other parts from the one you have and swap them - I don't recall how they attach but it should be obvious when you're holding the new one.
Personally, I couldn't get that thing off my downtube fast enough... first thing I did when my car got home. I put MiniFini's on instead - much more functional and less ugly... now I'm using custom cupholders I fabbed out of black PVC pipe and caps that plug into the small OEM holders and expand/extend them to a similar size and position as the MiniFini, but are quickly removable.
