There's a chip called TP4056 that's quite good for charging li-ion/li-po cells. You can buy the chip really cheap, or like above - a finished board with usb power in for $1.50 (!) on ebay including shipping. I bought one, soldered on a couple of silicon wires with magnets attached to the end. Also added a plug-able tiny voltmeter (ebay, but this other one from fasttech also looks quite good). I'm not sure how the voltmeter affects the charging (as it feeds off the charging current), but I recon it should be quite fine.
Per default it uses 1 amp of charging current, which is quite good for large cells, but you can modify it by changing one of the onboard resistors - lower values and you can use it for smaller cells safely.
It's quite a decent little charger, it manages pretty close to true CC/CV (constant current / constant voltage). It does get a bit hot when charging at 1 amp, especially in the start when the voltage difference between input and output is large, but nothing too worrying.
Termination seems to happen according to spec - just around 4.20 volt.
All in all - quite happy with it. I recon the chip (or even the board) could easily be built into projects that need li-ion/po charging, or even into a battery pack. Can't beat the price! :)