spark sql createOrReplaceTempView registerTempTable

spark sql ,如果每个表里的数据都很大,用sql直接将几个表关联起来查询时比较耗内存。可以先将每个表的可用集合过滤到最小,然后在内存中关联,可以提高减少内存占用,提高效率。
createOrReplaceTempView2.x版本以上。
registerTempTable1.5.x
  val data1 = dataSelect1(sqlContext, sparkModel)  val data2 = dataSelect2(sqlContext, sparkModel)  data1.createOrReplaceTempView("new_table1_info")  data2.createOrReplaceTempView("new_table2_info")   val sql='select t1.*,t2.* from new_table1_info t1 left join new_table2_info t2 on t1.name=t2.name2' where t1.age>18 ;  val result = sqlContext.sql(sql)   def dataSelect1(sqlContext: HiveContext, sm: SparkModel): DataFrame = {    val sql = "select name,age,other  from table1 where name='' "    sqlContext.sql(sql)  }     def dataSelect2(sqlContext: HiveContext, sm: SparkModel): DataFrame = {    val sql = "select name2,age2,other2,hh  from table2 where name='' "    sqlContext.sql(sql)  }
计算机