
Elder was one of four players to leave the club with Ben Osborn, Andi Weimann and Jake Rooney also not being offered new contracts after revealing their retained list last Friday.
Derry Murkin's arrival in January put Elder's position in jeopardy but the Australian leaves with a promotion under his belt having helped the club finish second in League One in the 2023/24 season.
Former Derby coach Phillips worked with Elder at Leicester City where the left-back started his career which, after a series of loan spells, led him to Hull City and then Derby.
And the ex-Sunderland striker feels that from the club's perspective, it was the right call.
"I know Callum and he’s a fantastic lad," Phillips told BestBettingSites.
"I had him at Leicester when I was a coach and watched his career develop, and he’s a very good player, but he suffered with a couple of injuries late on, towards the back end of the season.
"He’s 31 years of age so I can kind of almost get where the football club are coming from.
"From a sentimental point of view, it’s a tough one for the player.
"I can see why the fans have a mixed reaction, but from the football club, from a football decision, probably, it’s the right decision."
Elder was first-choice at the beginning of the season, but missed the back end of the campaign due to a fractured toe and then underwent minor knee surgery.
Derby fans were unable to give him a proper send-off but in a message to supporters about his exit he said: “It’s been a privilege and honour to represent this football club for the last three years.
“To achieve another promotion and help the club progress over the last 12 months has been a special part of my career.
“Pride Park is an incredible place to play football and I’m truly grateful I got the opportunity to experience that.
“Thank you for your amazing support and I wish you all nothing but success in the future.”
Every day, the DerbyshireLive football desk strives to deliver all manner of news, features and transfer-related stories as a part of our overall package of Derby County content.
Our dedicated reporter Leigh Curtis - who you can follow on X by clicking here - follows the Rams home and away and offers you comprehensive coverage from matchdays, press conferences and everything that happens in between.
You can also get all your favourite content from DerbyshireLive's Rams team on WhatsApp. Click here to sign up for breaking updates about the biggest stories in and around the club.
If you prefer reading our Rams stories on your phone, consider downloading the DerbyshireLive app, in which you can personalise the content you see by selecting Derby County as one of your designated topics. You can get it from Apple here and for Android here.