Can I Change How Free Shipping Class Displays on the WooCommerce Cart Page?

Yes, WordPress is open-source, so it possible to change each bit in it. To change the Free Shipping Class, you first need to create one.

Go to WooCommerce > Settings > Shipping > Shipping Classes

Now after you created the Shipping Class, assign this class to any shipping method and product. Then paste the following code in your functions.php file

function wbcom_free_shipping_label( $label, $method ) {
if ( 0 == $method->cost ) {
$label = 'Free shipping!';
}
return $label;
}
add_filter( 'woocommerce_cart_shipping_method_full_label', 'wbcom_free_shipping_label', 10, 2 );

In the above code, you can assign any value in $label variable.

Now check up on the product which is assigned to this class. The Free Shipping label should be changed.