我创建了一个 spark 程序，它从 Hive 数据库中检索 table 名称，然后计算单个 table 的行数。但是，我试图通过获取多个 tables 的行数来提高这一点，从 2 个 tables 开始。

我的火花代码是：

from pyspark.sql import SparkSession
import sys

def sql_count_rows(db,table):
    sql_query = """select count(*) from {0}.{1}""".format(db,table)
    return sql_query

db_name = sys.argv[1]

spark = SparkSession \
    .builder \
    .appName("HiveTableRecordCount") \
    .enableHiveSupport() \
    .getOrCreate()

spark.sql("use {0}".format(db_name))
tables_df=spark.sql("show tables").collect()
tables_df=tables_df[0:2] #filter the first two tables
print("list content: ",tables_df)
print("list length: ",len(tables_df))
queryBuilder=""
#queryBuilder=queryBuilder + """select count(*) from {0}.{1}""".format(tables_df['database'], tables_df['tableName'])
#print("queryBuilder: ",queryBuilder)
loop_length=1
index=0
while loop_length < len(tables_df):
    queryBuilder =  sql_count_rows(tables_df[index]['database'], tables_df[index]['tableName'])#tables_df.foreach(lambda row: sql_count_rows(row,queryBuilder))
    queryBuilder = queryBuilder + "\nunion all \n"
    loop_length+=1
    index+=1

spark.sql(queryBuilder).show()
spark.stop()

为了编写我受 https://tunetotech.com/Post.aspx?post=8 文章启发的代码，用 scala spark 编写。

当我执行程序时，我收到以下错误：

pyspark.sql.utils.ParseException: u"\nmismatched input '' 期待 {'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'MAP', 'REDUCE'}（第 3 行，位置 0）\n\n== SQL ==\nselect count(*) from gr_mapping.active_contracts_stg_v2\nunion all \n^^^\n"

你能帮忙理解我做错了什么吗？

queryBuilder = queryBuilder + " union all "```