laravel的批量插入或更新

在項目中常常有些需求是需要將大量的數據導入庫中，如果庫中不存在該條數據插入，存在則更新，典型應用場景：更新報表數據，有這些報表的數據歸因時間長達28天，也就是28內的數據都會更新，每天還會產生新的數據，這時就需要對新的數據插入，老數據進行更新。在laravel中有批量插入，批量更新的方法，也有對單條數據的插入或更新方法，卻沒有對批量數據的插入或更新的方法。

需求：

批量插入數據，如果表中存在則更新。

實現條件：

在表中設置唯一索引，通過唯一索引判斷該條數據在庫中是否存在
如有created_at字段則設置該字段爲TIMESPACE ，默認爲：CURRENT_TIMESTAMP （插入數據時不填該字段，默認值爲當前時間）

原理：

sql語句：

    insert into ads_daily_campaign_report 
        (apple_id ,date ,campaign_id ,campaign ,installs ,spent ,updated_at  ) 
    values 
        ( '591888' , '2019-11-11' , '123456' , 'campaign_name_1' , '12' , '34' , '2019-08-11 05:54:03'  ) ,
        ( '591888' , '2019-11-11' , '123457' , 'campaign_name_2' , '123' , '344' , '2019-08-11 05:54:03'  ) 
    on duplicate key update apple_id = 
        values (apple_id) ,date = values (date) ,campaign_id = values (campaign_id) ,campaign = values (campaign) ,installs = values (installs) ,spent = values (spent) ,updated_at = values (updated_at) ;

實現方法

/**
 * 批量插入或更新表中數據
 *
 * @param $data 要插入的數據，元素中的key爲表中的column，value爲對應的值
 * @param string $table 要插入的表
 * @param array $columns 要更新的的表的字段 
 * @return array
 */
public static function batchInsertOrUpdate($data,$table = '',$columns = []){

        if(empty($data)){//如果傳入數據爲空 則直接返回
            return [
                'insertNum' => 0,
                'updateNum' => 0
            ];
        }

        //拼裝sql
        $sql = "insert into ".$table." (";
        foreach ($columns as $k => $column) {
            $sql .= $column ." ,";
        }
        $sql = trim($sql,',');
        $sql .= " ) values ";

        foreach ($data as $k => $v){
            $sql .= "(";
            foreach ($columns as $kk => $column){
                if('updated_at' == $column){ //如果庫中存在，create_at字段會被更新
                    $sql .= " '".date('Y-m-d H:i:s')."' ,";
                }else{
                    $val = ''; //插入數據中缺少$colums中的字段時的默認值
                    if(isset($v[$column])){
                        $val = $v[$column];
                        $val = addslashes($val);  //在預定義的字符前添加反斜槓的字符串。
                    }
                    $sql .= " '".$val."' ,";
                }
            }
            $sql = trim($sql,',');
            $sql .= " ) ,";
        }
        $sql = trim($sql,',');
        $sql .= "on duplicate key update ";
        foreach ($columns as $k => $column){
            $sql .= $column ." = values (".$column.") ,";
        }
        $sql = trim($sql,',');
        $sql .= ';';
        
        $columnsNum = count($data);
        $retNum = DB::update(DB::raw($sql));
        $updateNum = $retNum - $columnsNum;
        $insertNum = $columnsNum - $updateNum;
        return [
            'insertNum' => $insertNum,
            'updateNum' => $updateNum
        ];
    }

建表語句

CREATE TABLE `myLaravel`.`ads_daily_campaign_report` (
  `id` INT NOT NULL AUTO_INCREMENT,
  `apple_id` INT NOT NULL DEFAULT 0,
  `date` DATETIME NOT NULL,
  `campaign_id` INT NOT NULL DEFAULT 0,
  `campaign` VARCHAR(45) NOT NULL,
  `installs` INT NOT NULL DEFAULT 0,
  `spent` DOUBLE NOT NULL DEFAULT 0.00,
  `created_at` TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP,
  `updated_at` TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  UNIQUE INDEX `apple_id-date-campaign_id` (`apple_id` ASC, `date` ASC, `campaign_id` ASC));

注：記得加唯一索引

執行語句

    $data = [
        ['apple_id' => '591888','date' => '2019-11-11','campaign_id' => '123456','campaign' => 'campaign_name_1','installs' => '12', 'spent' => '34'],
        ['apple_id' => '591888','date' => '2019-11-11','campaign_id' => '123457','campaign' => 'campaign_name_2','installs' => '123', 'spent' => '344'],
        ];
    $table = 'ads_daily_campaign_report';
    $columns = ['apple_id','date','campaign_id','campaign','installs','spent','updated_at'];
  
    $ret = BaseModel::batchInsertOrUpdate($data,$table,$columns);
    dd($ret);

執行結果：

運行結果：
    array:2 [▼
  "insertNum" => 2
  "updateNum" => 0
]

mysql 查詢結果

mysql> SELECT * FROM myLaravel.ads_daily_campaign_report;
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
| id | apple_id | date                | campaign_id | campaign        | installs | spent | created_at          | updated_at          |
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
|  1 |   591888 | 2019-11-11 00:00:00 |      123456 | campaign_name_1 |       12 |    34 | 2019-08-11 16:40:42 | 2019-08-11 08:40:42 |
|  2 |   591888 | 2019-11-11 00:00:00 |      123457 | campaign_name_2 |      123 |   344 | 2019-08-11 16:40:42 | 2019-08-11 08:40:42 |
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
2 rows in set (0.00 sec)

面向對象的方式實現

上大學時教軟件工程的老師,在上課時非常嚴肅的問過我們一個問題，至今記憶猶新。“你們知道是什麼人在推動這個世界進步麼？” “科學家、天才、瘋子……”我們思考良久也沒有回答正確這個問題，最後還是老師給了我們一個令人信服的答案：懶人。

思考：

在調用時每次都要輸入表名和表的字段，很麻煩，還容易出錯，這是不是有優化空間呢？表名和表的字段也基本是一一對應的（特殊情況可能只更新個別字段），用面向對象的方式好像剛好可以解決這個問題，下面用面向對象的思想解決這個問題。

思想：

基類：實現具體方法
子類：設置屬性，調用方法，

代碼實現：

基類：

<?php

namespace App\Model;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\DB;

class BaseModel extends Model{

    /**
     * 與模型關聯的表名
     *
     * @var string
     */
    protected $table = '';

    /**
     * 用來向表中插入數據的字段
     *
     * @var array
     */
    protected $tableColumns = [];

    /**
     * 批量插入或更新表中數據
     *
     * @param $data 要插入的數據，元素中的key爲表中的column，value爲對應的值
     * @param string $table 要插入的表
     * @param array $columns 要更新的的表的字段 
     * @return array
     */
    public function batchInsertOrUpdate($data,$table = '',$columns = []){

        if(empty($data)){//如果傳入數據爲空 則直接返回
            return [
                'insertNum' => 0,
                'updateNum' => 0
            ];
        }

        empty($table) && $table = $this->getTable();  //如果未傳入table則通過對象獲得
        empty($columns) && $columns = $this->getTableColumns();  //如果未傳入table則通過對象獲得

        //拼裝sql
        $sql = "insert into ".$table." (";
        foreach ($columns as $k => $column) {
            $sql .= $column ." ,";
        }
        $sql = trim($sql,',');
        $sql .= " ) values ";

        foreach ($data as $k => $v){
            $sql .= "(";
            foreach ($columns as $kk => $column){
                if('updated_at' == $column){ //如果庫中存在，create_at字段會被更新
                    $sql .= " '".date('Y-m-d H:i:s')."' ,";
                }else{
                    $val = ''; //插入數據中缺少$colums中的字段時的默認值
                    if(isset($v[$column])){
                        $val = $v[$column];
                        $val = addslashes($val);  //在預定義的字符前添加反斜槓的字符串。
                    }
                    $sql .= " '".$val."' ,";
                }
            }
            $sql = trim($sql,',');
            $sql .= " ) ,";
        }
        $sql = trim($sql,',');
        $sql .= "on duplicate key update ";
        foreach ($columns as $k => $column){
            $sql .= $column ." = values (".$column.") ,";
        }
        $sql = trim($sql,',');
        $sql .= ';';

        $columnsNum = count($data);
        $retNum = DB::update(DB::raw($sql));
        $updateNum = $retNum - $columnsNum;
        $insertNum = $columnsNum - $updateNum;
        return [
            'insertNum' => $insertNum,
            'updateNum' => $updateNum
        ];
    }

    /**
     * 返回表中字段
     *
     * @return array
     */
    public function getTableColumns(){
        if(empty($this->tableColumns)){
            return [];
        }
        return $this->tableColumns;
    }
}

子類：

<?php

namespace App\Model;


class AdsDailyCampaignReport extends BaseModel{

    /**
     * 與模型關聯的表名
     *
     * @var string
     */
    protected $table = 'ads_daily_campaign_report';

    /**
     * 用來向表中插入數據的字段
     *
     * @var array
     */
    protected $tableColumns = [
        'apple_id',
        'date',
        'campaign_id',
        'campaign',
        'installs',
        'spent',
        'updated_at'
    ];

}

<?php

namespace App\Model;


class AdsDailyCountryCampaignReport extends BaseModel{

    /**
     * 與模型關聯的表名
     *
     * @var string
     */
    protected $table = 'ads_daily_country_campaign_report';

    /**
     * 用來向表中插入數據的字段
     *
     * @var array
     */
    protected $tableColumns = [
        'apple_id',
        'date',
        'country',
        'campaign_id',
        'campaign',
        'installs',
        'spent',
        'updated_at'
    ];

}

建表語句

CREATE TABLE `myLaravel`.`ads_daily_country_campaign_report` (
  `id` INT NOT NULL AUTO_INCREMENT,
  `apple_id` INT NOT NULL DEFAULT 0,
  `date` DATETIME NOT NULL,
  `country` VARCHAR(2) NOT NULL,
  `campaign_id` INT NOT NULL DEFAULT 0,
  `campaign` VARCHAR(45) NOT NULL,
  `installs` INT NOT NULL DEFAULT 0,
  `spent` DOUBLE NOT NULL DEFAULT 0.00,
  `created_at` TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP,
  `updated_at` TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`id`),
  UNIQUE INDEX `apple_id-date-country-campaign_id` (`apple_id` ASC, `date` ASC,`country` ASC, `campaign_id` ASC));

執行語句：

$data = [
    ['apple_id' => '591888','date' => '2019-11-12','campaign_id' => '123456','campaign' => 'campaign_name_1','installs' => '12', 'spent' => '34'],
    ['apple_id' => '591888','date' => '2019-11-11','campaign_id' => '123457','campaign' => 'campaign_name_2','installs' => '1231', 'spent' => '3441'],
        ];

    $adsDailyCampaign = new AdsDailyCampaignReport();
    $ret = $adsDailyCampaign->batchInsertOrUpdate($data);
    dd($ret);

執行結果：

array:2 [▼
  "insertNum" => 1
  "updateNum" => 1
]

mysql> SELECT * FROM myLaravel.ads_daily_campaign_report;
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
| id | apple_id | date                | campaign_id | campaign        | installs | spent | created_at          | updated_at          |
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
|  1 |   591888 | 2019-11-11 00:00:00 |      123456 | campaign_name_1 |       12 |    34 | 2019-08-11 16:40:42 | 2019-08-11 08:40:42 |
|  2 |   591888 | 2019-11-11 00:00:00 |      123457 | campaign_name_2 |     1231 |  3441 | 2019-08-11 16:40:42 | 2019-08-11 09:26:39 |
|  3 |   591888 | 2019-11-12 00:00:00 |      123456 | campaign_name_1 |       12 |    34 | 2019-08-11 17:26:39 | 2019-08-11 09:26:39 |
+----+----------+---------------------+-------------+-----------------+----------+-------+---------------------+---------------------+
3 rows in set (0.00 sec)

執行語句：

$data = [
    ['apple_id' => '591888','date' => '2019-11-12','country' => 'US','campaign_id' => '123456','campaign' => 'campaign_name_1','installs' => '12', 'spent' => '34'],
    ['apple_id' => '591888','date' => '2019-11-11','country' => 'CN','campaign_id' => '123457','campaign' => 'campaign_name_2','installs' => '1231', 'spent' => '3441'],
        ];

    $adsDailyCountryCampaign = new AdsDailyCountryCampaignReport();
    $ret = $adsDailyCountryCampaign->batchInsertOrUpdate($data);
    dd($ret);

執行結果：

array:2 [▼
  "insertNum" => 2
  "updateNum" => 0
]

mysql> SELECT * FROM myLaravel.ads_daily_country_campaign_report;
+----+----------+---------------------+---------+-------------+-----------------+----------+-------+---------------------+---------------------+
| id | apple_id | date                | country | campaign_id | campaign        | installs | spent | created_at          | updated_at          |
+----+----------+---------------------+---------+-------------+-----------------+----------+-------+---------------------+---------------------+
|  1 |   591888 | 2019-11-12 00:00:00 | US      |      123456 | campaign_name_1 |       12 |    34 | 2019-08-11 17:33:32 | 2019-08-11 09:33:32 |
|  2 |   591888 | 2019-11-11 00:00:00 | CN      |      123457 | campaign_name_2 |     1231 |  3441 | 2019-08-11 17:33:32 | 2019-08-11 09:33:32 |
+----+----------+---------------------+---------+-------------+-----------------+----------+-------+---------------------+---------------------+
2 rows in set (0.01 sec)

laravel的批量插入或更新

laravel的批量插入或更新

需求：

實現條件：

原理：

實現方法

建表語句

執行語句

執行結果：

mysql 查詢結果

面向對象的方式實現

思考：

思想：

代碼實現：

建表語句

執行語句：

執行結果：

執行語句：

執行結果：

工作中用到的腳本合集

微服務實踐Aspire項目發佈到遠程k8s集羣

通過f-string編寫簡潔高效的Python格式化輸出代碼

[轉帖]20個常用的Linux工具命令

[轉帖]PostgreSQL從小白到高手教程 - 第46講：poc-tpch測試

24-5-18 X

閱讀筆記 > 關於代碼註釋

閱讀筆記 > 靠“巧合”編程？

閱讀筆記 > 編程需要考慮的問題

laravel的批量插入或更新

閱讀筆記 > 受歡迎程序的設計特點

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結