While LLM-as-a-Judge offers a favorable alternative to human evaluations, closed source LLMs impose some limitations with this evaluation framework.