An affine transformation t is given by some square matrix a
and some vector b, and maps x to a * x + b.
 One can represent such a transformation t by an augmented matrix,
whose first n columns are those of a and whose last column has
the entries of b. We also denote this matrix by t.
 Then the n first columns represent the linear part a of the
transformation t, and its last column represents the translation
part, the vector b.
 Since the transformation t maps a vector x to a * x + b,
provided a is invertible, with inverse A, then so is t,
and its inverse T maps x to A * x - A * b.
 Explanation: if y = t(x), then y = a * x + b, so
y - b = a * x, and if a is invertible with inverse A,
then A * y + A * (-b) = A * a * x, which means
x =A * y + A * (-b), so the linear part isA
and the translation part isA * (-b)`.
 Therefore the matrix for t_inv, also denoted as T, has
A as its first n columns and A * (-b) as its last column.
 Here is an implementation of a function that returns the inverse of an
affine transformation. The function includes a documentation string which
summarizes the discussion above, and an example based on the one in
your question.
 def affine_inverse(t, as_block_matrix=False):
    """
    Return the inverse of this affine transformation
    The affine transformation is given by some `n` by `(n + 1)`
    matrix `t` whose `n` first columns represent the linear
    part `a` of the transformation, and whose last column represents
    the vector `b`, so that the transformation maps a vector `x`
    to `a * x + b`. Provided `a` is invertible, with inverse `A`,
    then so is `t`, and its inverse `T` maps `x` to `A * x - A * b`.
    Optionally, `T` is returned as a block matrix.
    EXAMPLE::
        sage: k = GF(3)
        sage: alist = [
        ....: [1, 0, 0, 2, 0, 2],
        ....: [2, 2, 1, 1, 2, 0],
        ....: [2, 0, 0, 2, 2, 2],
        ....: [2, 2, 0, 1, 1, 1],
        ....: [1, 0, 2, 0, 2, 0],
        ....: [0, 0, 0, 2, 0, 2],
        ....: ]
        sage: blist = (1, 1, 0, 1, 0, 2)
        sage: a = matrix(k, alist)
        sage: n = a.nrows()
        sage: b = matrix(k, n, 1, blist)
        sage: t = a.augment(b)
        sage: affine_inverse(t)
        [1 0 0 0 0 2 1]
        [1 0 2 2 0 2 2]
        [2 0 1 0 2 0 1]
        [2 1 0 2 1 0 1]
        [2 0 2 0 0 2 0]
        [1 2 0 1 2 2 1]
        sage: affine_inverse(t, as_block_matrix=True)
        [1 0 0 0 0 2|1]
        [1 0 2 2 0 2|2]
        [2 0 1 0 2 0|1]
        [2 1 0 2 1 0|1]
        [2 0 2 0 0 2|0]
        [1 2 0 1 2 2|1]
    """
    k = t.base_ring()
    n = t.nrows()
    a = t.delete_columns([n])
    b = t.delete_columns(range(n))
    A = ~a
    if as_block_matrix:
        return block_matrix(k, 1, 2, [A, A * -b])
    return A.augment(A * -b)
 
Can you provide an example of an affine transformation you are interested in?
I have edited my question with an example, any help would be appreciated.
Do you mean like a projective coordinate representation? In that case if you have
Mx+byou could make a block matrix out ofMandblikeblock_matrix([[M,b],[0,1]])or something, and then get that inverse. But I don't know if this is what you mean by the matrix representation of an affine transformation.