The left-back quit Glanford Park on a free transfer during the summer but has found first-team opportunities limited at the Madejski Stadium.
He has made five appearances in all competitions for the Royals but has not played since August.
Williams also enjoyed a period on loan at League One outfit Peterborough earlier in the season, making three appearances.
Source: PA
Source: PA