It's called 23A or A23 and you are going to need 6 of them for each box or 12 for the pair.
Each battery is 12v and would cost around $4 each. You may find difficult to buy all 12 of them (for the pair) at 1 local CVS or Walgreen store because they might not have all avail at 1 time.
Many people bought the A23 batteries from ebay. It's cheaper and with free delivery.