You are exactly right - that is how my cable failed. The arc swing of the lever wears an oval hole in the end, and eventually the thin plastic sheath inside to allow the steel cable to slide in the steel conduit can begin to poke out the end. I guess the ovalized wear acts as a slight funnel, so the plastic sheath pokes out, but then jams going back in, causing the cable to hang up. It'd be nice if the plastic sheath were glued in, so it could not migrate.
My other thought is if the plate where the cable terminates allowed the flexible conduit to pivot ... or just buy a new cable every 30,000 miles.