I was a die-hard fan until they got swept out of the playoffs again in 2010. I probably missed a handful of Baker's starts his entire Twins career. I'm qualified to make an informed opinion on him. I saw a pitcher who benefited from playing in the weak Central with the unbalanced schedule. I saw a pitcher who cracked any time there was pressure involved. Even 163 against Detroit he pitched well when they were behind until the Twins scored a couple runs. Then Scotty went out for the top of the inning and gave it right back. Baker and pressure were never a good mix.
If we don't use the eye test but instead use stats, we may question Pelfrey's mental toughness by the 5.59 ERA he had in the first inning last year, his ability to only get ten total outs after the sixth inning last year or the 30 second soap opera that takes place between each of his pitches.
Those things also show up on the eye test. I doubt Baker is more mentally weak than Pelfrey, but hypothetically if he was, I'd still take the guy who gives up fewer runs, strikes more guys out and goes deeper into games.