The visitors took the lead through Marc Richards' third goal in two games but Kedwell equalised early in the second-half with his third penalty of the campaign.
Port Vale fired a warning shot when Doug Loft crossed to the back post where Adam Yates nodded back across goal for Tom Pope whose header hit the bar.
Pope then turned provider for Vale's opener in the 33rd-minute when he flicked on a long ball to Richards who poked home.
The Gills equalised with 63 minutes gone when Frank Nouble was brought down by Rob Taylor and although Kedwell's first effort was saved, a re-take was ordered and he made no mistake second time around.
Vale goalkeeper Stuart Tomlinson denied the Gills a winner five minutes from time when he kept out Matt Fish's fierce drive.