Any chance you can insert a drill bit that is smaller than the piece you are trying to remove? Wedge it in there and spray a small amount of WD40 to reduce any friction, then using pliers turn the bit to a point of engaging the banana piece the pull on the drill bit. If any movement is witnessed then repeat the process except the WD40 and it should work out a little at a time.
if you use the rat tail file make sure to insert the tang in as the file is designed to cut on the push stroke and won’t engage the broken piece as you pull.
good luck