The problem of rpm is usually caused by leaking/ malfunction of the vacuum hoses, as you are saying that they are connected make sure that they are connected properly in correct places.
Also does this also happen on load? I.e when headlights etc are turned on?